821 papers
Training Methods
0/819Optimizers, schedulers, and training infrastructure.
Progress0 of 819
2026
195- MayMolmoAct2: Action Reasoning Models for Real-world Deploymentagents2605.02881Allen Institute for AIMay 4, 2026~118 min
- MayToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agentsagents2605.12481TongyiLabMay 12, 2026~122 min
- MayPREPING: Building Agent Memory without Tasksagents2605.13880KAIST AIMay 11, 2026~94 min
- MayOrchard: An Open-Source Agentic Modeling Frameworkagents2605.15040Microsoft ResearchMay 14, 2026~130 min
- MayHINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agentsagents2605.17873KAIST AIMay 18, 2026~112 min
- MaySkillOpt: Executive Strategy for Self-Evolving Agent Skillsagents2605.23904Microsoft ResearchMay 22, 2026~125 min
- MayIt Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMsalignment2605.20258KAIST AIMay 18, 2026~122 min
- MayConditional Equivalence of DPO and RLHF: Implicit Assumption, Failure Modes, and Provable Alignmentalignment2605.20834May 20, 2026~103 min
- MaySlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-trainingarchitecture2605.08738May 9, 2026~106 min
- Mayoptimize_anything: A Universal API for Optimizing any Text Parameterarchitecture2605.19633May 19, 2026~111 min
- MayGated DeltaNet-2: Decoupling Erase and Write in Linear Attentionarchitecture2605.22791NVIDIAMay 21, 2026~109 min
- MayContext Training with Active Information Seekingcontext-optimization2605.13050DeepmindMay 13, 2026~123 min
- MayPlanningBench: Generating Scalable and Verifiable Planning Data for Evaluating and Training Large Language Modelsdata2605.20873Tencent HunyuanMay 20, 2026~120 min
- MayLens: Rethinking Training Efficiency for Foundational Text-to-Image Modelsdiffusion2605.21573Microsoft ResearchMay 20, 2026~117 min
- MayFrom Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skillsevaluation2605.23899Microsoft ResearchMay 22, 2026~120 min
- MayFull Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Stepsinference-optimization2605.16928RTP-LLMMay 16, 2026~132 min
- MayContinuous Latent Diffusion Language Modelllm-systems2605.06548ByteDance SeedMay 7, 2026~125 min
- MayReinforcing Multimodal Reasoning Against Visual Degradationmultimodal2605.09262Tencent HunyuanMay 10, 2026~119 min
- MayTraining Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Contextmultimodal2605.13831ByteDance SeedMay 13, 2026~98 min
- MayLearning from Language Feedback via Variational Policy Distillationreasoning2605.15113Salesforce AI ResearchMay 14, 2026~108 min
- MayAEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learningrl-training2605.00425BAIDUMay 1, 2026~125 min
- MayRebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVRrl-training2605.10781Microsoft ResearchMay 11, 2026~103 min
- MayThe Unlearnability Phenomenon in RLVR for Language Modelsrl-training2605.16787May 16, 2026~129 min
- MayEnd-to-End Autoregressive Image Generation with 1D Semantic Tokenizertraining-methods2605.00503ByteDance SeedMay 1, 2026~115 min
- MayListwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplextraining-methods2605.06139Tencent HunyuanMay 7, 2026~112 min
- MayContinuous-Time Distribution Matching for Few-Step Diffusion Distillationtraining-methods2605.06376alibaba-incMay 7, 2026~110 min
- MayDeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verificationtraining-methods2605.09269Tencent HunyuanMay 10, 2026~113 min
- MayAdaptive Teacher Exposure for Self-Distillation in LLM Reasoningtraining-methods2605.11458ByteDanceMay 12, 2026~124 min
- MayLearning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillationtraining-methods2605.11739Tencent HunyuanMay 12, 2026~113 min
- MayEndPrompt: Efficient Long-Context Extension via Terminal Anchoringtraining-methods2605.14589BAIDUMay 14, 2026~127 min
- MaySee What I Mean: Aligning Vision and Language Representations for Video Fine-grained Object Understandingtraining-methods2605.18018TongyiLabMay 18, 2026~118 min
- MayLiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMsvision2605.17260DeepmindMay 17, 2026~106 min
- AprCORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discoveryagents2604.01658Massachusetts Institute of TechnologyApr 2, 2026score 9~131 min
- AprAgent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligenceagents2604.18292ByteDance SeedApr 20, 2026~108 min
- AprToward Scalable Terminal Task Synthesis via Skill Graphsagents2604.25727Tencent HunyuanApr 28, 2026~130 min
- AprInCoder-32B-Thinking: Industrial Code World Model for Thinkingcode2604.03144Apr 3, 2026score 9~100 min
- AprAttention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigationcontext-optimization2604.10098LongCatApr 11, 2026score 9~127 min
- AprElucidating the SNR-t Bias of Diffusion Probabilistic Modelsdiffusion2604.16044alibaba-incApr 17, 2026~105 min
- AprDMax: Aggressive Parallel Decoding for dLLMsinference-optimization2604.08302National University of SingaporeApr 9, 2026~108 min
- AprStochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharinginference-optimization2604.22782AppleApr 3, 2026~101 min
- AprLeveraging Verifier-Based Reinforcement Learning in Image Editingllm-systems2604.27505ByteDance SeedApr 30, 2026~121 min
- AprRethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capabilityreasoning2604.06628AI45ResearchApr 8, 2026score 9~124 min
- AprDual-View Training for Instruction-Following Information Retrievalretrieval2604.18845SnowflakeApr 20, 2026~95 min
- AprLearning to Hint for Reinforcement Learningrl-training2604.00698SnowflakeApr 1, 2026score 9~123 min
- AprUnifying Group-Relative and Self-Distillation Policy Optimization via Sample Routingrl-training2604.02288Apr 2, 2026score 10~101 min
- AprModeling Multiple Support Strategies within a Single Turn for Emotional Support Conversationsrl-training2604.17972Qwen DianJinApr 20, 2026~118 min
- AprWorld-R1: Reinforcing 3D Constraints for Text-to-Video Generationrl-training2604.24764Microsoft ResearchApr 27, 2026~117 min
- AprHow Fast Should a Model Commit to Supervision? Training Reasoning Models on the Tsallis Loss Continuumrl-training2604.25907GoogleApr 28, 2026~131 min
- AprAccelerating RL Post-Training Rollouts via System-Integrated Speculative Decodingrl-training2604.26779NVIDIAApr 29, 2026~133 min
- AprA Survey of On-Policy Distillation for Large Language Modelstraining-methods2604.00626Apr 1, 2026~136 min
- AprSandMLE: A Multi-Agent Framework that Generates Synthetic Sandbox Environments for Training Machine Learning Engineering Agentstraining-methods2604.04872AI at MetaApr 6, 2026score 9~97 min
- AprRAGEN-2: Reasoning Collapse in Agentic RLtraining-methods2604.06268MLL LabApr 7, 2026score 9~118 min
- AprFP4 Explore, BF16 Train: Diffusion Reinforcement Learning via Efficient Rollout Scalingtraining-methods2604.06916NVIDIAApr 8, 2026~116 min
- AprContinuous Adversarial Flow Modelstraining-methods2604.11521ByteDance SeedApr 13, 2026score 8~119 min
- AprSelf-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervisiontraining-methods2604.12002Princeton UniversityApr 13, 2026score 9~122 min
- AprLightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillationtraining-methods2604.13010NVIDIAApr 14, 2026~132 min
- AprRethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipetraining-methods2604.13016OpenBMBApr 14, 2026score 10~136 min
- AprLeapAlign: Post-Training Flow Matching Models at Any Generation Step by Building Two-Step Trajectoriestraining-methods2604.15311ByteDance SeedApr 16, 2026score 8~110 min
- AprThe Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillationtraining-methods2604.16830Salesforce AI ResearchApr 18, 2026~115 min
- AprTCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agentstraining-methods2604.24005TongyiLabApr 27, 2026~112 min
- AprHY-Embodied-0.5: Embodied Foundation Models for Real-World Agentsuncategorized2604.07430Tencent HunyuanApr 8, 2026score 9~127 min
- AprELT: Elastic Looped Transformers for Visual Generationvision2604.09168DeepmindApr 10, 2026score 9~111 min
- MarLLM2Vec-Gen: Generative Embeddings from Large Language Modelsagents2603.10913McGill NLP GroupMar 11, 2026~94 min
- MarTRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemasagents2603.16448meituanMar 17, 2026score 9~98 min
- MarProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agentsagents2603.18815NVIDIAMar 19, 2026~99 min
- MarA Subgoal-driven Framework for Improving Long-Horizon LLM Agentsagents2603.19685DeepmindMar 20, 2026score 10~135 min
- MarCharacterFlywheel: Scaling Iterative Improvement of Engaging and Steerable LLMs in Productionalignment2603.01973Meta LlamaMar 2, 2026score 9~140 min
- MarAttention Residualsarchitecture2603.15031Moonshot AIMar 16, 2026score 9~110 min
- MarMixture-of-Depths Attentionarchitecture2603.15619ByteDance SeedMar 16, 2026score 9~111 min
- MarPolyglot-Lion: Efficient Multilingual ASR for Singapore via Balanced Fine-Tuning of Qwen3-ASRarchitecture2603.16184Knovel EngineeringMar 17, 2026score 3~116 min
- MarPOLCA: Stochastic Generative Optimization with LLMcode2603.14769DeepmindMar 16, 2026score 9~113 min
- MarKernel-Smith: A Unified Recipe for Evolutionary Kernel Optimizationcode2603.28342NVIDIAMar 30, 2026~139 min
- MarREAD MOREinference-optimization2603.05451Together AIMar 5, 2026~113 min
- MarUnified Spatio-Temporal Token Scoring for Efficient Video VLMsinference-optimization2603.18004Ai2Mar 18, 2026score 9~114 min
- MarTAPS: Task Aware Proposal Distributions for Speculative Samplinginference-optimization2603.27027Image and Video Understanding LabMar 27, 2026score 9~125 min
- MarSparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsitylow-precision2603.05168Microsoft ResearchMar 5, 2026score 9~79 min
- MarFrom Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoningmultimodal2603.03825QwenMar 4, 2026score 9~124 min
- MarSpatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Trainingmultimodal2603.12255Tencent HunyuanMar 12, 2026score 9~139 min
- MarIntern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scalemultimodal2603.25040InternLM / Shanghai AI LabMar 26, 2026~113 min
- MardaVinci-LLM:Towards the Science of Pretrainingpretraining2603.27164SII-GAIRMar 28, 2026score 9~110 min
- MarBeyond Length Scaling: Synergizing Breadth and Depth for Generative Reward Modelsreasoning2603.01571Tencent HunyuanMar 2, 2026score 9~108 min
- MarLongCat-Flash-Prover: Advancing Native Formal Reasoning via Agentic Tool-Integrated Reinforcement Learningreasoning2603.21065Meituan LongCatMar 22, 2026score 9~110 min
- MarWhy Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?reasoning2603.24472Microsoft ResearchMar 25, 2026score 9~109 min
- MarLearning to Retrieve from Agent Trajectoriesretrieval2604.04949RUC-GSAI-IIRLabMar 30, 2026score 9~118 min
- MarBreaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Modelsrl-training2603.07777Microsoft ResearchMar 8, 2026score 9~119 min
- Mar$V_{0.5}$: Generalist Value Model as a Prior for Sparse RL Rolloutsrl-training2603.10848LongCatMar 11, 2026score 9~123 min
- MarAI Can Learn Scientific Tasterl-training2603.14473OpenMOSSMar 15, 2026~105 min
- MarComplementary Reinforcement Learningrl-training2603.17621alibaba-incMar 18, 2026score 9~129 min
- MarFIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimizationrl-training2603.19835QwenMar 20, 2026score 10~109 min
- MarPivotRL: High Accuracy Agentic Post-Training at Low Compute Costrl-training2603.21383NVIDIAMar 22, 2026~123 min
- MarOn the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitationrl-training2603.22117QwenMar 23, 2026score 9~121 min
- MarSparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMsrl-training2603.22446QwenMar 23, 2026score 9~112 min
- MarUniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generationrl-training2603.23500ByteDance SeedMar 24, 2026score 9~101 min
- MarLearn Hard Problems During RL with Reference Guided Fine-tuningtraining-methods2603.01223ByteDance SeedMar 1, 2026score 9~119 min
- MarLaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrievaltraining-methods2603.01425Alibaba DAMOMar 2, 2026score 2~116 min
- MarRethinking Training Targets, Architectures and Data Quality for Universal Speech Enhancementtraining-methods2603.02641Mar 3, 2026~97 min
- MarScaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspacestraining-methods2603.06713Microsoft ResearchMar 5, 2026score 9~112 min
- MarHY-WU (Part I): An Extensible Functional Neural Memory Framework and An Instantiation in Text-Guided Image Editingtraining-methods2603.07236Tencent HunyuanMar 7, 2026score 6~135 min
- MarIn-Context Reinforcement Learning for Tool Use in Large Language Modelstraining-methods2603.08068National University of SingaporeMar 9, 2026~114 min
- MarHow Far Can Unsupervised RLVR Scale LLM Training?training-methods2603.08660Tsinghua UniversityMar 9, 2026~126 min
- MarMM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Datatraining-methods2603.09206NVIDIAMar 10, 2026score 9~112 min
- MarMeta-Reinforcement Learning with Self-Reflection for Agentic Searchtraining-methods2603.11327Ai2Mar 11, 2026~97 min
- MarSupervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Modelstraining-methods2603.13985AmazonMar 14, 2026~117 min
- MarRepresentation Alignment for Just Image Transformers is not Easier than You Thinktraining-methods2603.14366KAIST AIMar 15, 2026score 8~128 min
- MarOnline Experiential Learning for Language Modelstraining-methods2603.16856Microsoft ResearchMar 17, 2026score 9~121 min
- MarNemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillationtraining-methods2603.19220NVIDIAMar 19, 2026~94 min
- MarBeyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMDtraining-methods2603.20155DeepmindMar 20, 2026score 9~109 min
- MarRoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Modelstraining-methods2603.21341KAIST AIMar 22, 2026score 9~108 min
- MarManifold-Aware Exploration for Reinforcement Learning in Video Generationtraining-methods2603.21872Tencent HunyuanMar 23, 2026score 9~110 min
- MarSpatialBoost: Enhancing Visual Representation through Language-Guided Reasoningtraining-methods2603.22057KAIST AIMar 23, 2026score 6~115 min
- MarUnderstanding the Challenges in Iterative Generative Optimization with LLMstraining-methods2603.23994DeepmindMar 25, 2026score 9~94 min
- MarMolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulationuncategorized2603.16861Ai2Mar 17, 2026score 9~120 min
- MarHelios: Real Real-Time Long Video Generation Modelvision2603.04379NVIDIAMar 4, 2026score 9~109 min
- MarSAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editingvision2603.19228BAIDUMar 19, 2026score 3~105 min
- FebSWE-Universe: Scale Real-World Verifiable Environments to Millionsagents2602.02361QwenFeb 2, 2026score 4~109 min
- FebSEAD: Self-Evolving Agent for Multi-Turn Service Dialogueagents2602.03548meituanFeb 3, 2026score 5~110 min
- FebProAct: Agentic Lookahead in Interactive Environmentsagents2602.05327Tencent HunyuanFeb 5, 2026score 8~95 min
- FebAgentCPM-Report: Interleaving Drafting and Deepening for Open-Ended Deep Researchagents2602.06540OpenBMBFeb 6, 2026score 9~98 min
- FebScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Trainingagents2602.06820LongCatFeb 6, 2026score 9~103 min
- FebAgent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learningagents2602.10090SnowflakeFeb 10, 2026score 9~111 min
- FebWebWorld: A Large-Scale World Model for Web Agent Trainingagents2602.14721Qwen / Alibaba CloudFeb 16, 2026score 9~101 min
- FebWhy Steering Works: Toward a Unified View of Language Model Parameter Dynamicsalignment2602.02343alibabaFeb 2, 2026score 4~107 min
- FebOutcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Modelsalignment2602.04649QwenFeb 4, 2026score 9~109 min
- FebRecurrent-Depth VLA: Implicit Test-Time Compute Scaling of Vision-Language-Action Models via Latent Iterative Reasoningarchitecture2602.07845Ai2Feb 8, 2026score 6~107 min
- FebMiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modelingarchitecture2602.11761Feb 12, 2026~104 min
- FebDr. Kernel: Reinforcement Learning Done Right for Triton Kernel Generationscode2602.05885Feb 5, 2026~111 min
- FebOPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iterationdata2602.05400QwenFeb 5, 2026score 7~121 min
- FebData Science and Technology Towards AGI Part I: Tiered Data Managementdata2602.09003OpenBMBFeb 9, 2026score 8~115 min
- FebComposition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Modelsdata2602.12036Tencent HunyuanFeb 12, 2026score 5~100 min
- FebRoboCurate: Harnessing Diversity with Action-Verified Neural Trajectory for Robot Learningdata2602.18742KAIST AIFeb 21, 2026score 2~101 min
- FebOn Data Engineering for Scaling LLM Terminal Capabilitiesdata2602.21193NVIDIAFeb 24, 2026score 3~102 min
- FebArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillationdiffusion2602.09014Fudan UniversityFeb 9, 2026score 8~99 min
- FebSpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuninginference-optimization2602.13515Feb 13, 2026~108 min
- FebTest-Time Training with KV Binding Is Secretly Linear Attentionllm-systems2602.21204NVIDIAFeb 24, 2026score 9~131 min
- FebERNIE 5.0 Technical Reportmultimodal2602.04705Feb 4, 2026~138 min
- FebDreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generationmultimodal2602.12160ByteDanceFeb 12, 2026score 3~121 min
- FebA Mixed Diet Makes DINO An Omnivorous Vision Encodermultimodal2602.24181DeepmindFeb 27, 2026score 6~109 min
- FebSPARKLING: Balancing Signal Preservation and Symmetry Breaking for Width-Progressive Learningpretraining2602.02472ByteDance SeedFeb 2, 2026score 2~112 min
- FebVideoWorld 2: Learning Transferable Knowledge from Real-world Videospretraining2602.10102ByteDance SeedFeb 10, 2026score 2~107 min
- FebOn the Optimal Reasoning Length for RL-Trained Language Modelsreasoning2602.09591DeepSeekFeb 10, 2026~137 min
- FebGood SFT Optimizes for SFT, Better SFT Prepares for Reinforcement Learningrl-training2602.01058Feb 1, 2026~103 min
- FebCoBA-RL: Capability-Oriented Budget Allocation for Reinforcement Learning in LLMsrl-training2602.03048LongCatFeb 3, 2026score 6~95 min
- FebiGRPO: Self-Feedback-Driven LLM Reasoningrl-training2602.09000NVIDIAFeb 9, 2026~120 min
- FebWorldCompass: Reinforcement Learning for Long-Horizon World Modelsrl-training2602.09022Tencent HunyuanFeb 9, 2026score 8~114 min
- FebD-CORE: Incentivizing Task Decomposition in Large Reasoning Models for Complex Tool Usetraining-methods2602.02160alibaba-incFeb 2, 2026score 9~101 min
- FebTraining LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalabilitytraining-methods2602.02477Microsoft ResearchFeb 2, 2026score 8~105 min
- FebSelf-Hinting Language Models Enhance Reinforcement Learningtraining-methods2602.03143Feb 3, 2026~121 min
- FebLate-to-Early Training: LET LLMs Learn Earlier, So Faster and Bettertraining-methods2602.05393ByteDance SeedFeb 5, 2026score 8~112 min
- FebReinforcement World Model Learning for LLM-based Agentstraining-methods2602.05842Microsoft ResearchFeb 5, 2026score 3~88 min
- FebCanzona: A Unified, Asynchronous, and Load-Balanced Framework for Distributed Matrix-based Optimizerstraining-methods2602.06079QwenFeb 4, 2026score 5~119 min
- FebSeeUPO: Sequence-Level Agentic-RL with Convergence Guaranteestraining-methods2602.06554Tongyi-MAIFeb 6, 2026score 9~111 min
- FebF-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Raretraining-methods2602.06717T-TechFeb 6, 2026score 9~113 min
- FebPhyCritic: Multimodal Critic Models for Physical AItraining-methods2602.11124NVIDIAFeb 11, 2026score 6~105 min
- FebABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learningtraining-methods2602.11236Alibaba AMAP CV LabFeb 11, 2026score 2~111 min
- FebLearning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolationtraining-methods2602.12125Tencent HunyuanFeb 12, 2026score 9~91 min
- Febjina-embeddings-v5-text: Task-Targeted Embedding Distillationtraining-methods2602.15547Feb 17, 2026~106 min
- FebWorld Guidance: World Modeling in Condition Space for Action Generationtraining-methods2602.22010ByteDance SeedFeb 25, 2026score 2~102 min
- FebGLM-5: from Vibe Coding to Agentic Engineeringuncategorized2602.15763Zhipu / GLMFeb 17, 2026~148 min
- FebWorld Action Models are Zero-shot Policiesuncategorized2602.15922NVIDIA Deep Imagination ResearchFeb 17, 2026score 2~101 min
- FebMode Seeking meets Mean Seeking for Fast Long Video Generationvision2602.24289NVIDIAFeb 27, 2026score 2~106 min
- JanNitroGen: An Open Foundation Model for Generalist Gaming Agentsagents2601.02427NVIDIAJan 4, 2026score 2~125 min
- JanVLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memoryagents2601.08665ByteDance SeedJan 13, 2026score 3~124 min
- JanUnlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Textagents2601.10355LongCatJan 15, 2026score 9~111 min
- JanEvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experienceagents2601.15876meituanJan 22, 2026score 9~120 min
- JanComputer Environments Elicit General Agentic Intelligence in LLMsagents2601.16206Microsoft ResearchJan 22, 2026score 5~94 min
- JanDeep Delta Learningarchitecture2601.00417Jan 1, 2026~93 min
- JanPost-LayerNorm Is Back: Stable, ExpressivE, and Deeparchitecture2601.19895ByteDance SeedJan 27, 2026score 5~108 min
- JanX-Coder: Advancing Competitive Programming with Fully Synthetic Tasks, Solutions, and Testscode2601.06953Microsoft ResearchJan 11, 2026score 8~97 min
- JanStable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Modelcode2601.15892ByteDance SeedJan 22, 2026score 8~142 min
- JanMotion Attribution for Video Generationdata2601.08828NVIDIAJan 13, 2026score 2~115 min
- JanDreamStyle: A Unified Framework for Video Stylizationdiffusion2601.02785ByteDanceJan 6, 2026score 2~109 min
- JanMoEBlaze: Breaking the Memory Wall for Efficient MoE Training on Modern GPUsdistributed-training2601.05296Jan 8, 2026~113 min
- JanFine-tuning Small Language Models as Efficient Enterprise Search Relevance Labelersevaluation2601.03211Jan 6, 2026~113 min
- JanTransition Matching Distillation for Fast Video Generationinference-optimization2601.09881NVIDIAJan 14, 2026score 9~115 min
- JanFP8-RL: A Practical and Stable Low-Precision Stack for LLM Reinforcement Learninglow-precision2601.18150NVIDIAJan 26, 2026score 9~102 min
- JanNextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generationmultimodal2601.02204ByteDanceJan 5, 2026score 9~124 min
- JanDIFFA-2: A Practical Diffusion Large Language Model for General Audio Understandingmultimodal2601.23161LongCatJan 30, 2026score 5~125 min
- JanThe Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoningreasoning2601.06002ByteDanceJan 9, 2026score 8~152 min
- JanMultiplex Thinking: Reasoning via Token-wise Branch-and-Mergereasoning2601.08808Microsoft ResearchJan 13, 2026score 9~92 min
- JanDistribution-Aligned Sequence Distillation for Superior Long-CoT Reasoningreasoning2601.09088Alibaba Cloud Apsara Lab Jan 14, 2026score 6~88 min
- JanGDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationrl-training2601.05242Jan 8, 2026~109 min
- JanYour Group-Relative Advantage Is Biasedrl-training2601.08521Jan 13, 2026~113 min
- JanJet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flowrl-training2601.14243NVIDIAJan 20, 2026score 9~104 min
- JanLess Noise, More Voice: Reinforcement Learning for Reasoning via Instruction Purificationrl-training2601.21244BAIDUJan 29, 2026score 8~98 min
- JanVAR RL Done Right: Tackling Asynchronous Policy Conflicts in Visual Autoregressive Generationtraining-methods2601.02256ByteDanceJan 5, 2026score 5~115 min
- JanHow Do Large Language Models Learn Concepts During Continual Pre-Training?training-methods2601.03570Jan 7, 2026score 4~121 min
- JanChaining the Evidence: Robust Reinforcement Learning for Deep Search Agents with Citation-Aware Rubric Rewardstraining-methods2601.06021Z.aiJan 9, 2026score 9~112 min
- JanTranslateGemma Technical Reporttraining-methods2601.09012Google ResearchJan 13, 2026~107 min
- JanUncertainty-Aware Gradient Signal-to-Noise Data Selection for Instruction Tuningtraining-methods2601.13697alibaba-incJan 20, 2026score 9~129 min
- JanDenseGRPO: From Sparse to Dense Reward for Flow Matching Model Alignmenttraining-methods2601.20218TongyiLabJan 28, 2026score 6~100 min
- JanQwen3-ASR Technical Reporttraining-methods2601.21337Qwen / Alibaba CloudJan 29, 2026score 2~125 min
- JanQuartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimationtraining-methods2601.22813NVIDIAJan 30, 2026score 9~112 min
- JanGolden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Texttraining-methods2601.22975NVIDIAJan 30, 2026score 9~105 min
- JanDecouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-trainingtraining-methods2602.00747Jan 31, 2026~121 min
- JanCosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planninguncategorized2601.16163NVIDIAJan 22, 2026score 2~115 min
2025
241- DecSeed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experienceagents2512.17260ByteDance SeedDec 19, 2025score 8~129 min
- DecGR-Dexter Technical Reportagents2512.24210ByteDance SeedDec 30, 2025score 2~103 min
- DecLet It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystemagents2512.24873alibaba-incDec 31, 2025score 9~130 min
- DecStronger Normalization-Free Transformersarchitecture2512.10938Dec 11, 2025~114 min
- DecDynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Spacearchitecture2512.24617ByteDance SeedDec 31, 2025score 9~113 min
- DecmHC: Manifold-Constrained Hyper-Connectionsarchitecture2512.24880DeepSeekDec 31, 2025score 5~125 min
- DecFew-Step Distillation for Text-to-Image Generation: A Practical Guidediffusion2512.13006Alibaba DAMODec 15, 2025score 6~86 min
- DecImage Diffusion Preview with Consistency Solverdiffusion2512.13592DeepmindDec 15, 2025score 7~99 min
- DecEnd-to-End Training for Autoregressive Video Diffusion via Self-Resamplingdiffusion2512.15702ByteDance SeedDec 17, 2025score 10~101 min
- DecLLaDA2.0: Scaling Up Diffusion Language Models to 100Bdiffusion2512.15745Dec 10, 2025~122 min
- DecFast and Accurate Causal Parallel Decoding using Jacobi Forcinginference-optimization2512.14681Dec 16, 2025score 9~104 min
- DecReasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMsinference-optimization2512.17206alibaba-incDec 19, 2025score 9~117 min
- DecDeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelsllm-systems2512.02556DeepSeekDec 2, 2025~127 min
- DecSonicMoE: Accelerating MoE with IO and Tile-aware Optimizationsmoe2512.14080Dec 16, 2025~105 min
- DecCoupling Experts and Routers in Mixture-of-Experts via an Auxiliary Lossmoe2512.23447Dec 29, 2025~111 min
- DecOmni-Attribute: Open-vocabulary Attribute Encoder for Visual Concept Personalizationmultimodal2512.10955Snap ResearchDec 11, 2025score 5~100 min
- DecSpatialTree: How Spatial Abilities Branch Out in MLLMsmultimodal2512.20617ByteDance SeedDec 23, 2025score 8~112 min
- DecNemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Modelsreasoning2512.13607NVIDIADec 15, 2025score 9~112 min
- DecStabilizing Reinforcement Learning with LLMs: Formulation and Practicesrl-training2512.01374Dec 1, 2025~126 min
- DecGR-RL: Going Dexterous and Precise for Long-Horizon Robotic Manipulationrl-training2512.01801ByteDance SeedDec 1, 2025score 2~116 min
- DecJustRL: Scaling a 1.5B LLM with a Simple RL Reciperl-training2512.16649OpenBMBDec 18, 2025~120 min
- DecEvaluating Parameter Efficient Methods for RLVRrl-training2512.23165DeepSeekDec 29, 2025~105 min
- DecSpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RLtraining-methods2512.04069NVIDIADec 3, 2025score 7~118 min
- DecOn the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelstraining-methods2512.07783CMU-LTIDec 8, 2025score 9~115 min
- DecQwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Managementtraining-methods2512.12967Dec 15, 2025~110 min
- DecDirectional Textual Inversion for Personalized Text-to-Image Generationtraining-methods2512.13672KAIST AIDec 15, 2025score 5~102 min
- DecEfficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedtraining-methods2512.14067NVIDIADec 16, 2025score 9~107 min
- DecNemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode Supervisiontraining-methods2512.15489NVIDIADec 17, 2025score 8~103 min
- DecExploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Rewardtraining-methods2512.16912Columbia UniversityDec 18, 2025score 9~105 min
- DecFaithLens: Detecting and Explaining Faithfulness Hallucinationtraining-methods2512.20182Tsinghua NLP GroupDec 23, 2025score 6~102 min
- DecMasking Teacher and Reinforcing Student for Distilling Vision-Language Modelstraining-methods2512.22238NVIDIADec 23, 2025score 9~102 min
- DecDiversity or Precision? A Deep Dive into Next Token Predictiontraining-methods2512.22955Dec 28, 2025~107 min
- DecEnd-to-End Test-Time Training for Long Contexttraining-methods2512.23675Dec 29, 2025~120 min
- DecCosmos-H-Surgical: Learning Surgical Robot Policies from Videos via World Modelinguncategorized2512.23162NVIDIADec 29, 2025score 2~99 min
- NovScaling Agent Learning via Experience Synthesisagents2511.03773Nov 5, 2025~120 min
- NovVirtual Width Networksarchitecture2511.11238ByteDance SeedNov 14, 2025score 9~112 min
- NovNemotron-Flash: Towards Latency-Optimal Hybrid Small Language Modelsarchitecture2511.18890NVIDIANov 24, 2025score 9~106 min
- NovAdversarial Flow Modelsdiffusion2511.22475ByteDance SeedNov 27, 2025score 3~110 min
- NovDecoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shielddiffusion2511.22677Tongyi-MAINov 27, 2025score 4~114 min
- NovLlama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasksllm-systems2511.07025NVIDIANov 10, 2025score 9~99 min
- NovNVIDIA Nemotron Nano V2 VLmultimodal2511.03929NVIDIANov 6, 2025score 9~122 min
- NovMusic Flamingo: Scaling Music Understanding in Audio Language Modelsmultimodal2511.10289NVIDIANov 13, 2025score 6~121 min
- NovHunyuanOCR Technical Reportmultimodal2511.19575Tencent HunyuanNov 24, 2025score 9~146 min
- NovHarmony: Harmonizing Audio and Video Generation through Cross-Task Synergymultimodal2511.21579Tencent HunyuanNov 26, 2025score 5~107 min
- NovThe Path Not Taken: RLVR Provably Learns Off the Principalsrl-training2511.08567AI at MetaNov 11, 2025score 9~110 min
- NovWMPO: World Model-based Policy Optimization for Vision-Language-Action Modelsrl-training2511.09515ByteDance SeedNov 12, 2025score 5~128 min
- NovSoft Adaptive Policy Optimizationrl-training2511.20347QwenNov 25, 2025score 9~108 min
- NovFP8-Flow-MoE: A Casting-Free FP8 Recipe without Double Quantization Errortraining-methods2511.02302Nov 4, 2025~122 min
- NovReinforcement Learning Improves Traversal of Hierarchical Knowledge in LLMstraining-methods2511.05933Meta AINov 8, 2025score 9~116 min
- NovDRIVE: Data Curation Best Practices for Reinforcement Learning with Verifiable Reward in Competitive Code Generationtraining-methods2511.06307OpenAINov 9, 2025score 8~114 min
- NovBlack-Box On-Policy Distillation of Large Language Modelstraining-methods2511.10643Nov 13, 2025~100 min
- NovBeat the long tail: Distribution-Aware Speculative Decoding for RL Trainingtraining-methods2511.13841Together AINov 17, 2025~98 min
- NovNemotron Elastic: Towards Efficient Many-in-One Reasoning LLMstraining-methods2511.16664NVIDIANov 20, 2025score 9~127 min
- NovSSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Spacetraining-methods2511.20102Nov 25, 2025score 9~129 min
- NovToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestrationtraining-methods2511.21689NVIDIANov 26, 2025score 9~106 min
- NovRevisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalizationtraining-methods2511.22586ByteDance SeedNov 27, 2025score 9~113 min
- NovVisual Spatial Tuningvision2511.05491ByteDance SeedNov 7, 2025score 9~119 min
- NovZ-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformervision2511.22699Tongyi-MAINov 27, 2025score 9~108 min
- OctLearning on the Job: An Experience-Driven Self-Evolving Agent for Long-Horizon Tasksagents2510.08002KnowledgeXLab@Shanghai AI LabOct 9, 2025score 9~110 min
- OctAgent Learning via Early Experienceagents2510.08558Oct 9, 2025~108 min
- OctDemystifying Reinforcement Learning in Agentic Reasoningagents2510.11701Oct 13, 2025score 9~127 min
- OctGame-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agentsagents2510.23691ByteDanceOct 27, 2025score 8~124 min
- OctWebLeaper: Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich Seekingagents2510.24697TongyiLabOct 28, 2025score 9~124 min
- OctAgentFold: Long-Horizon Web Agents with Proactive Context Managementagents2510.24699TongyiLabOct 28, 2025score 9~101 min
- OctTongyi DeepResearch Technical Reportagents2510.24701Oct 28, 2025~115 min
- OctBeyond Reasoning Gains: Mitigating General Capabilities Forgetting in Large Reasoning Modelsalignment2510.21978AI at MetaOct 24, 2025score 10~112 min
- OctVLA-0: Building State-of-the-Art VLAs with Zero Modificationarchitecture2510.13054NVIDIAOct 15, 2025score 6~100 min
- OctHigher-order Linear Attentionarchitecture2510.27258Oct 31, 2025~112 min
- OctCode Aesthetics with Agentic Reward Feedbackcode2510.23272Microsoft ResearchOct 27, 2025score 9~110 min
- OctWebscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levelsdata2510.06499SalesforceOct 7, 2025score 10~115 min
- OctAgentFrontier: Expanding the Capability Frontier of LLM Agents with ZPD-Guided Data Synthesisdata2510.24695TongyiLabOct 28, 2025score 9~113 min
- OctSelf-Forcing++: Towards Minute-Scale High-Quality Video Generationdiffusion2510.02283ByteDance SeedOct 2, 2025score 10~133 min
- OctTemporal Alignment Guidance: On-Manifold Sampling in Diffusion Modelsdiffusion2510.11057KAIST AIOct 13, 2025score 5~129 min
- OctAlphaFlow: Understanding and Improving MeanFlow Modelsdiffusion2510.20771SnapOct 23, 2025score 5~115 min
- OctSprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformersdiffusion2510.21986SnapOct 24, 2025score 9~110 min
- OctBoundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Modelsinference-optimization2510.11683Z.aiOct 13, 2025score 9~120 min
- OctINT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formatsinference-optimization2510.25602NVIDIAOct 29, 2025score 10~113 min
- OctEvery Activation Boosted: Scaling General Reasoner to 1 Trillion Open Language Foundationmoe2510.22115inclusionAIOct 25, 2025score 10~127 min
- OctSAIL-Embedding Technical Report: Omni-modal Embedding Foundation Modelmultimodal2510.12709ByteDanceOct 14, 2025score 9~151 min
- OctLongCat-Flash-Omni Technical Reportmultimodal2511.00279Meituan LongCatOct 31, 2025score 9~117 min
- OctAttention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimizationreasoning2510.13554alibaba-incOct 15, 2025score 9~117 min
- OctLoongRL: Reinforcement Learning for Advanced Reasoning over Long Contextsreasoning2510.19363Microsoft ResearchOct 22, 2025score 9~114 min
- OctUI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoningreasoning2510.20286TongyiLabOct 23, 2025score 6~107 min
- OctBroRL: Scaling Reinforcement Learning via Broadened Explorationrl-training2510.01180NVIDIAOct 1, 2025score 9~106 min
- OctLow-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Rewardrl-training2510.03222TencentOct 3, 2025score 9~107 min
- OctQeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMsrl-training2510.11696NVIDIAOct 13, 2025score 10~124 min
- OctThe Art of Scaling Reinforcement Learning Compute for LLMsrl-training2510.13786Oct 15, 2025~139 min
- OctLaSeR: Reinforcement Learning with Last-Token Self-Rewardingrl-training2510.14943Tencent HunyuanOct 16, 2025score 10~101 min
- OctDLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learningrl-training2510.15110OpenAIOct 16, 2025score 9~119 min
- OctRepurposing Synthetic Data for Fine-grained Search Agent Supervisionrl-training2510.24694TongyiLabOct 28, 2025score 9~105 min
- OctFine-Grained GRPO for Precise Preference Alignment in Flow Modelstraining-methods2510.01982IXCLab@Shanghai AI LabOct 2, 2025score 8~101 min
- OctWhy Low-Precision Transformer Training Fails: An Analysis on Flash Attentiontraining-methods2510.04212Tsinghua UniversityOct 5, 2025score 9~125 min
- OctScalable In-context Ranking with Generative Modelstraining-methods2510.05396DeepmindOct 6, 2025score 8~107 min
- OctDCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelismtraining-methods2510.10620Oct 12, 2025~99 min
- OctBitNet Distillationtraining-methods2510.13998Microsoft ResearchOct 15, 2025score 9~127 min
- OctEfficient Long-context Language Model Training by Core Attention Disaggregationtraining-methods2510.18121Oct 20, 2025score 10~105 min
- OctUnified Reinforcement and Imitation Learning for Vision-Language Modelstraining-methods2510.19307NVIDIAOct 22, 2025score 9~87 min
- OctDaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agentstraining-methods2510.19336OPPOOct 22, 2025score 8~104 min
- OctE2Rank: Your Text Embedding can Also be an Effective and Efficient Listwise Rerankertraining-methods2510.22733Alibaba DAMOOct 26, 2025score 9~108 min
- OctParallelMuse: Agentic Parallel Thinking for Deep Information Seekingtraining-methods2510.24698TongyiLabOct 28, 2025score 9~120 min
- OctPairUni: Pairwise Training for Unified Multimodal Language Modelstraining-methods2510.25682ByteDanceOct 29, 2025score 8~110 min
- OctDefeating the Training-Inference Mismatch via FP16training-methods2510.26788Sea AI LabOct 30, 2025score 9~127 min
- OctMemory Retrieval and Consolidation in Large Language Models through Function Tokensuncategorized2510.08203ByteDance SeedOct 9, 2025score 7~115 min
- OctSpatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learningvision2510.27606InternLM / Shanghai AI LabOct 31, 2025score 9~124 min
- SepThe Landscape of Agentic Reinforcement Learning for LLMs: A Surveyagents2509.02547Sep 2, 2025~127 min
- SepTowards General Agentic Intelligence via Environment Scalingagents2509.13311Sep 16, 2025~114 min
- SepLIMI: Less is More for Agencyagents2509.17567Sep 22, 2025~104 min
- SepScaling Generalist Data-Analytic Agentsagents2509.25084QwenSep 29, 2025score 9~103 min
- SepWhy Language Models Hallucinatealignment2509.04664Sep 4, 2025~118 min
- SepRLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewardsalignment2509.21319NVIDIASep 25, 2025score 9~105 min
- SepInverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?evaluation2509.04292ByteDance SeedSep 4, 2025score 6~97 min
- SepHunyuan-MT Technical Reportllm-systems2509.05209Tencent HunyuanSep 5, 2025score 3~99 min
- SepFast-dLLM v2: Efficient Block-Diffusion LLMllm-systems2509.26328Sep 30, 2025~114 min
- SepHunyuanImage 3.0 Technical Reportmoe2509.23951Tencent HunyuanSep 28, 2025score 9~121 min
- SepCapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learningmultimodal2509.22647InternLM / Shanghai AI LabSep 26, 2025score 6~114 min
- SepPre-training under infinite computepretraining2509.14786Sep 18, 2025~106 min
- SepPretraining Large Language Models with NVFP4pretraining2509.25149Sep 29, 2025~126 min
- SepNo Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shapingreasoning2509.21880KAIST AISep 26, 2025score 10~103 min
- SepQuantile Advantage Estimation: Stabilizing RLVR for LLM Reasoningreasoning2509.22611Sep 26, 2025score 9~119 min
- SepMeta-Awareness Enhances Reasoning Models: Self-Alignment Reinforcement Learningreasoning2510.03259KAIST AISep 26, 2025score 9~119 min
- SepSPARK: Synergistic Policy And Reward Co-Evolving Frameworkrl-training2509.22624InternLM / Shanghai AI LabSep 26, 2025score 9~110 min
- SepFantastic Pretraining Optimizers and Where to Find Themtraining-methods2509.02046Sep 2, 2025~121 min
- SepSharing is Caring: Efficient LM Post-Training with Collective RL Experience Sharingtraining-methods2509.08721DeepSeekSep 10, 2025score 10~108 min
- SepScaling Agents via Continual Pre-trainingtraining-methods2509.13310Sep 16, 2025~117 min
- SepSIM-CoT: Supervised Implicit Chain-of-Thoughttraining-methods2509.20317InternLM / Shanghai AI LabSep 24, 2025score 8~131 min
- SepWinning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuningtraining-methods2509.23873alibabaSep 28, 2025score 9~104 min
- SepSocratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolutiontraining-methods2509.24726alibaba-incSep 29, 2025score 9~105 min
- SepDeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Searchtraining-methods2509.25454Stanford NLPSep 29, 2025score 10~122 min
- SepTruthRL: Incentivizing Truthful LLMs via Reinforcement Learningtraining-methods2509.25760Sep 30, 2025~100 min
- SepKnapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocationtraining-methods2509.25849ByteDance SeedSep 30, 2025score 9~111 min
- SepMuon Outperforms Adam in Tail-End Associative Memory Learningtraining-methods2509.26030Sep 30, 2025~124 min
- SepFront-Loading Reasoning: The Synergy between Pretraining and Post-Training Datatraining-methods2510.03264NVIDIASep 26, 2025score 9~115 min
- SepCARE: Cognitive-reasoning Augmented Reinforcement for Emotional Support Conversationtraining-methods2510.05122Qwen DianJinSep 30, 2025score 1~100 min
- SepRobix: A Unified Model for Robot Interaction, Reasoning and Planninguncategorized2509.01106ByteDance SeedSep 1, 2025score 9~108 min
- AugFin-PRM: A Domain-Specialized Process Reward Model for Financial Reasoning in Large Language Modelsagents2508.15202Qwen DianJinAug 21, 2025score 9~111 min
- AugMemento: Fine-tuning LLM Agents without Fine-tuning LLMsagents2508.16153Aug 22, 2025score 9~114 min
- AugQwen-Image Technical Reportmultimodal2508.02324Qwen / Alibaba CloudAug 4, 2025~137 min
- AugIs Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lensreasoning2508.01191Aug 2, 2025~140 min
- AugNVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelreasoning2508.14444NVIDIAAug 20, 2025score 10~119 min
- AugBeyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVRrl-training2508.14029Aug 19, 2025score 9~128 min
- AugOn the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationtraining-methods2508.05629Aug 7, 2025score 9~107 min
- AugDuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimizationtraining-methods2508.14460ByteDance SeedAug 20, 2025score 9~112 min
- AugTreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modelingtraining-methods2508.17445ByteDance SeedAug 24, 2025score 9~105 min
- AugHermes 4 Technical Reporttraining-methods2508.18255Aug 25, 2025~110 min
- AugREINA: Regularized Entropy Information-Based Loss for Efficient Simultaneous Speech Translationuncategorized2508.04946Roblox CorporationAug 7, 2025score 2~114 min
- AugTowards Affordance-Aware Robotic Dexterous Grasping with Human-like Priorsuncategorized2508.08896DAMO AcademyAug 12, 2025score 2~120 min
- JulA Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligenceagents2507.21046Jul 28, 2025score 10~120 min
- JulWebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationdata2507.15061Alibaba DAMOJul 20, 2025score 9~109 min
- JulZeCO: Zero Communication Overhead Sequence Parallelism for Linear Attentiondistributed-training2507.01004Jul 1, 2025score 9~111 min
- JulFlexOlmo: Open Language Models for Flexible Data Usemoe2507.07024Jul 9, 2025~105 min
- JulThe Imitation Game: Turing Machine Imitator is Length Generalizable Reasonerreasoning2507.13332Intern Large ModelsJul 17, 2025score 9~106 min
- JulScaling RL to Long Videosrl-training2507.07966Jul 10, 2025~98 min
- JulGSPO: Towards Scalable Reinforcement Learning for Language Modelsrl-training2507.18071Qwen / Alibaba CloudJul 24, 2025~94 min
- JulAgentic Reinforced Policy Optimizationrl-training2507.19849Jul 26, 2025~95 min
- JulGeometric-Mean Policy Optimizationrl-training2507.20673Jul 28, 2025score 9~105 min
- JulThe Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigationsafety2507.05578Jul 8, 2025score 9~120 min
- JulAXLearn: Modular Large Model Training on Heterogeneous Infrastructuretraining-methods2507.05411AppleJul 7, 2025score 8~94 min
- JulToken Bottleneck: One Token to Remember Dynamicstraining-methods2507.06543NAVER AI LabJul 9, 2025score 5~116 min
- JulARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shortstraining-methods2507.20939TencentJul 28, 2025score 9~125 min
- JulGR-3 Technical Reportuncategorized2507.15493ByteDance SeedJul 21, 2025score 5~104 min
- JunLog-Linear Attentionarchitecture2506.04761Jun 5, 2025~122 min
- JunMultiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generationarchitecture2506.09991Jun 11, 2025score 9~114 min
- JunThe Automated LLM Speedrunning Benchmark: Reproducing NanoGPT Improvementscode2506.22419Jun 27, 2025score 10~118 min
- JunOpenThoughts: Data Recipes for Reasoning Modelsdata2506.04178DeepSeekJun 4, 2025score 9~121 min
- JunHunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialdiffusion2506.15442Tencent HunyuanJun 18, 2025score 2~104 min
- JunTransformers Meet In-Context Learning: A Universal Approximation Theoryinference-optimization2506.05200Jun 5, 2025~121 min
- JunLongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMsinference-optimization2506.14429Jun 17, 2025score 9~112 min
- JunMiniCPM4: Ultra-Efficient LLMs on End Devicesllm-systems2506.07900OpenBMBJun 9, 2025score 9~122 min
- JunGenRecal: Generation after Recalibration from Large to Small Vision-Language Modelsllm-systems2506.15681NVIDIAJun 18, 2025score 9~102 min
- JunQwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelspretraining2506.05176Qwen / Alibaba CloudJun 5, 2025score 10~82 min
- JunReinforcement Pre-Trainingpretraining2506.08007Jun 9, 2025~106 min
- JunMagistralreasoning2506.10910MistralJun 12, 2025~105 min
- JunMiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentionreasoning2506.13585DeepSeekJun 16, 2025score 9~132 min
- JunRLPR: Extrapolating RLVR to General Domains without Verifiersrl-training2506.18254OpenBMBJun 23, 2025score 9~83 min
- JunBeyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningtraining-methods2506.01939Jun 2, 2025score 9~123 min
- JunSelf Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusiontraining-methods2506.08009Jun 9, 2025score 9~120 min
- JunSparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsitytraining-methods2506.16500Jun 19, 2025score 9~103 min
- JunSeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Trainingvision2506.05301ByteDance SeedJun 5, 2025score 8~104 min
- MayVSA: Faster Video Diffusion with Trainable Sparse Attentiondiffusion2505.13389May 19, 2025score 9~105 min
- MaySageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Traininginference-optimization2505.11594May 16, 2025score 10~110 min
- MayModel Merging in Pre-training of Large Language Modelspretraining2505.12082May 17, 2025~98 min
- May100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelsreasoning2505.00551DeepSeekMay 1, 2025score 9~118 min
- MayLlama-Nemotron: Efficient Reasoning Modelsreasoning2505.00949DeepSeekMay 2, 2025~96 min
- MayQwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learningrl-training2505.17667May 23, 2025score 9~118 min
- MayParallel Scaling Law for Language Modelsscaling-laws2505.10475May 15, 2025~87 min
- MayRADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scaletraining-methods2505.03005May 5, 2025score 9~133 min
- MayZeroSearch: Incentivize the Search Capability of LLMs without Searchingtraining-methods2505.04588Alibaba DAMOMay 7, 2025~114 min
- MayA Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignmenttraining-methods2505.10717Microsoft ResearchMay 15, 2025~132 min
- MayG1: Bootstrapping Perception and Reasoning Abilities of Vision-Language Model via Reinforcement Learningtraining-methods2505.13426Moonshot AIMay 19, 2025score 9~102 min
- MayQuartet: Native FP4 Training Can Be Optimal for Large Language Modelstraining-methods2505.14669NVIDIAMay 20, 2025score 9~124 min
- MayFFT-based Dynamic Subspace Selection for Low-Rank Adaptive Optimization of Large Language Modelstraining-methods2505.17967Together AIMay 23, 2025~117 min
- MayThe Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelstraining-methods2505.22617May 28, 2025score 9~125 min
- MayCritical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Trainingtraining-methods2505.23971May 29, 2025~111 min
- AprBitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMsarchitecture2504.18415Apr 25, 2025score 9~90 min
- AprInternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelsmultimodal2504.10479Apr 14, 2025score 10~101 min
- AprEagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelsmultimodal2504.15271NVIDIAApr 21, 2025score 9~103 min
- AprKimi-Audio Technical Reportmultimodal2504.18425Moonshot AIApr 25, 2025score 9~104 min
- AprNemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-trainingpretraining2504.13161Apr 17, 2025score 10~127 min
- AprDianJin-R1: Evaluating and Enhancing Financial Reasoning in Large Language Modelsreasoning2504.15716Qwen DianJinApr 22, 2025score 9~106 min
- AprPhi-4-reasoning Technical Reportreasoning2504.21318Microsoft ResearchApr 30, 2025~123 min
- AprReinforcement Learning from Human Feedbackrl-training2504.12501Apr 16, 2025~84 min
- AprLearning to Reason under Off-Policy Guidancerl-training2504.14945Apr 21, 2025score 8~116 min
- AprTTRL: Test-Time Reinforcement Learningrl-training2504.16084Apr 22, 2025score 9~93 min
- AprInference-Time Scaling for Generalist Reward Modelingscaling-laws2504.02495Apr 3, 2025~101 min
- AprMinitron-SSM: Efficient Hybrid Language Model Compression through Group-Aware SSM Pruningtraining-methods2504.11409Apr 15, 2025score 9~124 min
- AprNEMOTRON-CROSSTHINK: Scaling Self-Learning beyond Math Reasoningtraining-methods2504.13941Apr 15, 2025~105 min
- AprReinforcement Learning for Reasoning in Large Language Models with One Training Exampletraining-methods2504.20571Apr 29, 2025score 10~117 min
- MarOn the Acquisition of Shared Grammatical Representations in Bilingual Language Modelsalignment2503.03962Mar 5, 2025score 4~111 min
- MarTransformers without Normalizationarchitecture2503.10622Mar 13, 2025~114 min
- MarRWKV-7 "Goose" with Expressive Dynamic State Evolutionarchitecture2503.14456Mar 18, 2025score 9~123 min
- MarGemma 3 Technical Reportarchitecture2503.19786Google ResearchMar 25, 2025~112 min
- MarHolistically Evaluating the Environmental Impact of Creating Language Modelsevaluation2503.05804Mar 3, 2025~121 min
- MarA Comprehensive Survey on Long Context Language Modelingllm-systems2503.17407Mar 20, 2025~121 min
- MarUnderstanding R1-Zero-Like Training: A Critical Perspectivereasoning2503.20783DeepSeekMar 26, 2025score 9~108 min
- MarOpen-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelreasoning2503.24290DeepSeekMar 31, 2025score 9~116 min
- MarDAPO: An Open-Source LLM Reinforcement Learning System at Scalerl-training2503.14476OpenAIMar 18, 2025~111 min
- MarTraining and Inference Efficiency of Encoder-Decoder Speech Modelstraining-methods2503.05931NVIDIAMar 7, 2025~114 min
- MarSearch-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningtraining-methods2503.09516Mar 12, 2025score 10~105 min
- MarTraining Video Foundation Models with NVIDIA NeMotraining-methods2503.12964NVIDIAMar 17, 2025score 6~127 min
- FebNative Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentionarchitecture2502.11089DeepSeekFeb 16, 2025score 9~127 min
- FebMoBA: Mixture of Block Attention for Long-Context LLMsarchitecture2502.13189Moonshot AIFeb 18, 2025score 9~112 min
- FebReformulation for Pretraining Data Augmentationdata2502.04235ByteDance SeedFeb 6, 2025score 9~95 min
- FebSmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelpretraining2502.02737Hugging Face Smol ModelsFeb 4, 2025score 10~133 min
- FebLIMO: Less is More for Reasoningreasoning2502.03387Feb 5, 2025~123 min
- FebProcess Reinforcement through Implicit Rewardsrl-training2502.01456Feb 3, 2025score 9~110 min
- FebLongRoPE2: Near-Lossless LLM Context Window Scalingscaling-laws2502.20082Feb 27, 2025score 10~92 min
- FebConverting MLPs into Polynomials in Closed Formtraining-methods2502.01032Feb 3, 2025~111 min
- FebExamining Two Hop Reasoning Through Information Content Scalingtraining-methods2502.03490Feb 5, 2025~117 min
- FebMatryoshka Quantizationtraining-methods2502.06786Feb 10, 2025score 9~112 min
- FebSWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutiontraining-methods2502.18449DeepSeekFeb 25, 2025score 9~116 min
- JanSFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingalignment2501.17161Jan 28, 2025score 10~103 min
- JanStreaming DiLoCo with overlapping communication: Towards a Distributed Free Lunchdistributed-training2501.18512Jan 30, 2025score 10~109 min
- JanrStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingreasoning2501.04519OpenAIJan 8, 2025score 9~115 min
- JanKimi k1.5: Scaling Reinforcement Learning with LLMsreasoning2501.12599Jan 22, 2025~107 min
- Jans1: Simple test-time scalingreasoning2501.19393OpenAIJan 31, 2025~112 min
- JanREINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalizationrl-training2501.03262Jan 4, 2025score 9~107 min
- JanDeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learningrl-training2501.12948DeepSeekJan 22, 2025score 10~129 min
- JanTransformer-Squared: Self-adaptive LLMstraining-methods2501.06252Jan 9, 2025score 9~115 min
- JanThe Lessons of Developing Process Reward Models in Mathematical Reasoningtraining-methods2501.07301Qwen / Alibaba CloudJan 13, 2025score 10~115 min
- JanOptimizing Large Language Model Training Using FP4 Quantizationtraining-methods2501.17116Jan 28, 2025score 9~107 min
- JanDecoding-based Regressiontraining-methods2501.19383DeepMindJan 31, 2025~128 min
2024
139- DecNVILA: Efficient Frontier Visual Language Modelsllm-systems2412.04468Dec 5, 2024~117 min
- DecNext Token Prediction Towards Multimodal Intelligence: A Comprehensive Surveymultimodal2412.18619Dec 16, 2024~124 min
- DecQwen2.5 Technical Reportpretraining2412.15115Qwen / Alibaba CloudDec 19, 2024~109 min
- Dec2 OLMo 2 Furiouspretraining2501.00656Allen Institute for AIDec 31, 2024~111 min
- DecTraining Large Language Models to Reason in a Continuous Latent Spacereasoning2412.06769Dec 9, 2024~126 min
- DecOffline Reinforcement Learning for LLM Multi-Step Reasoningreasoning2412.16145Dec 20, 2024score 9~122 min
- DecHunyuanProver: A Scalable Data Synthesis Framework and Guided Tree Search for Automated Theorem Provingreasoning2412.20735Dec 30, 2024score 9~119 min
- DecVLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Modelstraining-methods2412.01822NVIDIADec 2, 2024score 9~97 min
- DecAPOLLO: SGD-like Memory, AdamW-level Performancetraining-methods2412.05270Dec 6, 2024score 9~116 min
- DecNo More Adam: Learning Rate Scaling at Initialization is All You Needtraining-methods2412.11768Dec 16, 2024score 9~100 min
- DecMix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LNtraining-methods2412.13795Dec 18, 2024score 8~111 min
- DecRobustFT: Robust Supervised Fine-tuning for Large Language Models under Noisy Responsetraining-methods2412.14922Dec 19, 2024score 10~90 min
- DecLearnLM: Improving Gemini for Learningtraining-methods2412.16429Dec 21, 2024score 7~111 min
- DecB-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasonerstraining-methods2412.17256Dec 23, 2024score 9~118 min
- NovSparsing Law: Towards Large Language Models with Greater Activation Sparsityagents2411.02335Nov 4, 2024score 9~108 min
- NovSample-Efficient Alignment for LLMsalignment2411.01493Nov 3, 2024score 8~124 min
- NovWhen Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Trainingarchitecture2411.13476Nov 20, 2024score 8~114 min
- NovOPENCODER: THE OPEN COOKBOOK FOR TOP-TIER CODE LARGE LANGUAGE MODELScode2411.04905Nov 7, 2024~110 min
- NovHunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencentmoe2411.02265Tencent HunyuanNov 4, 2024score 9~110 min
- NovBalancing Pipeline Parallelism with Vocabulary Parallelismtraining-methods2411.05288Nov 8, 2024score 9~118 min
- NovCut Your Losses in Large-Vocabulary Language Modelstraining-methods2411.09009AppleNov 13, 2024score 9~114 min
- NovMARS: Unleashing the Power of Variance Reduction for Training Large Modelstraining-methods2411.10438Nov 15, 2024score 9~118 min
- NovTülu 3: Pushing Frontiers in Open Language Model Post-Trainingtraining-methods2411.15124Allen Institute for AINov 22, 2024~132 min
- NovDeMo: Decoupled Momentum Optimizationtraining-methods2411.19870Nov 29, 2024score 7~101 min
- OctSelf-Boosting Large Language Models with Synthetic Preference Dataalignment2410.06961Oct 9, 2024score 9~118 min
- OctAddition is All You Need for Energy-efficient Language Modelsinference-optimization2410.00907Oct 1, 2024score 9~109 min
- OctSeerAttention: Learning Intrinsic Sparse Attention in Your LLMsinference-optimization2410.13276Oct 17, 2024score 9~107 min
- OctA Survey of Small Language Modelsllm-systems2410.20011Oct 25, 2024~119 min
- OctEoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximationllm-systems2410.21271NVIDIAOct 28, 2024score 6~113 min
- OctPre-training Distillation for Large Language Models: A Design Space Explorationpretraining2410.16215Oct 21, 2024score 10~124 min
- OctRATIONALYST: Mining Implicit Rationales for Process Supervision of Reasoningreasoning2410.01044Oct 1, 2024score 9~110 min
- OctMA-RLHF: Reinforcement Learning from Human Feedback with Macro Actionsrl-training2410.02743BAIDUOct 3, 2024score 8~117 min
- OctAsynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Modelsrl-training2410.18252Oct 23, 2024score 9~117 min
- OctSwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformationserving2410.03960SnowflakeOct 4, 2024score 9~128 min
- OctREAD MOREserving2410.20399Together AIOct 27, 2024~108 min
- OctStuffed Mamba: Oversized States Lead to the Inability to Forgettraining-methods2410.07145Oct 9, 2024score 9~127 min
- OctLiger Kernel: Efficient Triton Kernels for LLM Trainingtraining-methods2410.10989Oct 14, 2024~124 min
- OctMerge to Learn: Efficiently Adding Skills to Language Models with Model Mergingtraining-methods2410.12937Oct 16, 2024~116 min
- OctUFT: Unifying Fine-Tuning of SFT and RLHF/DPO/UNA through a Generalized Implicit Reward Functiontraining-methods2410.21438Oct 28, 2024~111 min
- SepTowards a Unified View of Preference Learning for Large Language Models: A Surveyalignment2409.02795Sep 4, 2024score 10~110 min
- SepInstruction Following without Instruction Tuningalignment2409.14254Sep 21, 2024score 9~115 min
- SepIs Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysisalignment2409.20059Sep 30, 2024score 8~107 min
- SepConfigurable Foundation Models: Building LLMs from a Modular Perspectivearchitecture2409.02877OpenBMBSep 4, 2024score 9~129 min
- SepSource2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sourcesdata2409.08239Sep 12, 2024score 10~112 min
- SepMaskLLM: Learnable Semi-Structured Sparsity for Large Language Modelsllm-systems2409.17481Sep 26, 2024score 9~118 min
- SepOLMoE: Open Mixture-of-Experts Language Modelsmoe2409.02060Allen Institute for AISep 3, 2024~96 min
- SepNVLM: Open Frontier-Class Multimodal LLMsmultimodal2409.11402Sep 17, 2024~107 min
- SepMolmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Modelsmultimodal2409.17146Allen Institute for AISep 25, 2024score 9~121 min
- SepMM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-Tuningmultimodal2409.20566AppleSep 30, 2024~110 min
- SepScaling Smart: Accelerating Large Language Model Pre-training with Small Model Initializationpretraining2409.12903AppleSep 19, 2024score 9~102 min
- SepA Controlled Study on Long Context Extension and Generalization in LLMstraining-methods2409.12181Sep 18, 2024~146 min
- SepTraining Language Models to Self-Correct via Reinforcement Learningtraining-methods2409.12917Sep 19, 2024~103 min
- AugHermes 3 Technical Reportalignment2408.11857Aug 15, 2024~105 min
- AugAchieving Human Level Competitive Robot Table Tennisarchitecture2408.03906DeepMindAug 7, 2024score 2~120 min
- AugJamba-1.5: Hybrid Transformer-Mamba Models at Scalearchitecture2408.12570Aug 22, 2024~102 min
- AugFocusLLM: Precise Understanding of Long Context by Dynamic Condensingcontext-optimization2408.11745Aug 21, 2024score 9~135 min
- AugUnleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelsdata2408.02085Aug 4, 2024score 10~182 min
- AugSelf-Taught Evaluatorsevaluation2408.02666Aug 5, 2024score 9~103 min
- AugMUTUAL REASONING MAKES SMALLER LLMS STRONGER PROBLEM-SOLVERSreasoning2408.06195Aug 12, 2024~115 min
- AugDeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchreasoning2408.08152DeepSeekAug 15, 2024score 9~112 min
- AugThe Vizier Gaussian Process Bandit Algorithmtraining-methods2408.11527DeepMindAug 21, 2024~133 min
- AugLLM Pruning and Distillation in Practice: The Minitron Approachtraining-methods2408.11796Aug 21, 2024~110 min
- AugMemory-Efficient LLM Training with Online Subspace Descenttraining-methods2408.12857Aug 23, 2024score 7~126 min
- AugMulti-Layer Transformers Gradient Can be Approximated in Almost Linear Timetraining-methods2408.13233Aug 23, 2024score 9~100 min
- AugAUXILIARY-LOSS-FREE LOAD BALANCING STRATEGY FOR MIXTURE-OF-EXPERTStraining-methods2408.15664Aug 28, 2024~97 min
- AugPhysics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problemstraining-methods2408.16293Aug 29, 2024score 9~120 min
- AugBuilding and better understanding vision-language models: insights and future directionsvision2408.12637Aug 22, 2024score 10~96 min
- JulLearning to (Learn at Test Time): RNNs with Expressive Hidden Statesarchitecture2407.04620Jul 5, 2024~108 min
- JulLet the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Modelsmoe2407.01906DeepSeekJul 2, 2024score 9~107 min
- JulCompact Language Models via Pruning and Knowledge Distillationpretraining2407.14679Jul 19, 2024~140 min
- JulGemma 2: Improving Open Language Models at a Practical Sizepretraining2408.00118Google ResearchJul 31, 2024~106 min
- JulPhi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cyclesafety2407.13833Microsoft ResearchJul 18, 2024score 9~110 min
- JulScaling Granite Code Models to 128K Contexttraining-methods2407.13739IBM ResearchJul 18, 2024score 8~113 min
- JulEmbedding And Clustering Your Data Can Improve Contrastive Pretrainingtraining-methods2407.18887SnowflakeJul 26, 2024~101 min
- JunAligning Language Models with Demonstrated Feedbackalignment2406.00888Jun 2, 2024score 8~102 min
- JunBootstrapping Language Models with DPO Implicit Rewardsalignment2406.09760Jun 14, 2024score 9~116 min
- JunInstruction Pre-Training: Language Models are Supervised Multitask Learnerspretraining2406.14491Jun 20, 2024score 10~97 min
- JunWPO: Enhancing RLHF with Weighted Preference Optimizationrl-training2406.11827Zoom AIJun 17, 2024score 9~112 min
- JunAdam-mini: Use Fewer Learning Rates To Gain Moretraining-methods2406.16793Jun 24, 2024score 9~100 min
- MaySelf-Play Preference Optimization for Language Model Alignmentalignment2405.00675May 1, 2024score 9~96 min
- MayUnderstanding the performance gap between online and offline alignment algorithmsalignment2405.08448May 14, 2024score 9~107 min
- MayOpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Frameworkalignment2405.11143May 20, 2024score 9~99 min
- MayTransformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualityarchitecture2405.21060Amazon ScienceMay 31, 2024score 10~123 min
- MayDeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Datareasoning2405.14333DeepSeekMay 23, 2024score 9~101 min
- MayOffline Regularised Reinforcement Learning for Large Language Models Alignmentrl-training2405.19107May 29, 2024score 8~97 min
- MayRLHF Workflow: From Reward Modeling to Online RLHFtraining-methods2405.07863SalesforceMay 13, 2024score 10~96 min
- AprMixture-of-Depths: Dynamically allocating compute in transformer-based language modelsarchitecture2404.02258Apr 2, 2024score 9~101 min
- AprExtending Llama-3's Context Ten-Fold Overnightcontext-optimization2404.19553Apr 30, 2024score 9~103 min
- AprLayerSkip: Enabling Early Exit Inference and Self-Speculative Decodinginference-optimization2404.16710Apr 25, 2024score 9~107 min
- AprLLM2Vec: Large Language Models Are Secretly Powerful Text Encodersllm-systems2404.05961Apr 9, 2024~105 min
- AprRho-1: Not All Tokens Are What You Needpretraining2404.07965Microsoft ResearchApr 11, 2024score 9~96 min
- AprIterative Reasoning Preference Optimizationreasoning2404.19733Apr 30, 2024score 9~101 min
- AprChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipelinetraining-methods2404.02893Apr 3, 2024score 8~99 min
- AprReFT: Representation Finetuning for Language Modelstraining-methods2404.03592Apr 4, 2024~102 min
- AprStream of Search (SoS): Learning to Search in Languagetraining-methods2404.03683Apr 1, 2024score 9~121 min
- AprMiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiestraining-methods2404.06395OpenBMBApr 9, 2024score 9~139 min
- AprLearn Your Reference Model for Real Good Alignmenttraining-methods2404.09656Apr 15, 2024score 9~83 min
- AprBetter & Faster Large Language Models via Multi-token Predictiontraining-methods2404.19737Apr 30, 2024~103 min
- MarView publicationcode2403.13839AppleMar 14, 2024~98 min
- MarGecko: Versatile Text Embeddings Distilled from Large Language Modelsdata2403.20327DeepMindMar 29, 2024score 9~98 min
- MarDiPaCo: Distributed Path Compositiondistributed-training2403.10616DeepMindMar 15, 2024score 9~133 min
- MarBranch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLMmoe2403.07816Mar 12, 2024score 9~107 min
- MarSimple and Scalable Strategies to Continually Pre-train Large Language Modelspretraining2403.08763Mar 13, 2024score 9~127 min
- MarThe Unreasonable Ineffectiveness of the Deeper Layerspretraining2403.17887Mar 26, 2024score 9~100 min
- MarQuiet-STaR: Language Models Can Teach Themselves to Think Before Speakingreasoning2403.09629Mar 14, 2024score 9~107 min
- MarTeaching Large Language Models to Reason with Reinforcement Learningrl-training2403.04642Mar 7, 2024score 9~115 min
- MarGaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectiontraining-methods2403.03507Mar 6, 2024score 9~103 min
- MarLoHan: Low-Cost High-Performance Framework to Fine-Tune 100B Model on a Consumer GPUtraining-methods2403.06504Mar 11, 2024score 9~120 min
- MarRAFT: Adapting Language Model to Domain Specific RAGtraining-methods2403.10131Mar 15, 2024~103 min
- MarParameter Efficient Reinforcement Learning from Human Feedbacktraining-methods2403.10704Mar 15, 2024score 8~106 min
- MarInternLM2 Technical Reporttraining-methods2403.17297InternLM / Shanghai AI LabMar 26, 2024score 9~92 min
- FebKTO: Model Alignment as Prospect Theoretic Optimizationalignment2402.01306OpenBMBFeb 2, 2024~117 min
- FebODIN: Disentangled Reward Mitigates Hacking in RLHFalignment2402.07319Feb 11, 2024score 9~112 min
- FebSuppressing Pink Elephants with Direct Principle Feedbackalignment2402.07896EleutherAIFeb 12, 2024score 9~102 min
- FebMultilingual E5 Text Embeddings: A Technical Reportcontext-optimization2402.05672Feb 8, 2024~118 min
- FebLongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscontext-optimization2402.13753Feb 21, 2024~93 min
- FebSpeculative Streaming: Fast LLM Inference without Auxiliary Modelsinference-optimization2402.11131AppleFeb 16, 2024score 9~100 min
- FebThe Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicryllm-systems2402.04347Feb 6, 2024score 9~113 min
- FebLiRank: Industrial Large Scale Ranking Models at LinkedInllm-systems2402.06859Feb 10, 2024score 3~133 min
- FebWorld Model on Million-Length Video And Language With Blockwise RingAttentionllm-systems2402.08268Feb 13, 2024score 9~106 min
- FebOmniPred: Language Models as Universal Regressorspretraining2402.14547DeepMindFeb 22, 2024score 5~101 min
- FebResonance RoPE: Improving Context Length Generalization of Large Language Modelsreasoning2403.00071Feb 29, 2024score 9~100 min
- FebLearning to Learn Faster from Human Feedback with Language Model Predictive Controltraining-methods2402.11450DeepMindFeb 18, 2024score 9~104 min
- FebMegaScale: Scaling Large Language Model Training to More Than 10,000 GPUstraining-methods2402.15627Feb 23, 2024score 9~116 min
- FebTraining-Free Long-Context Scaling of Large Language Modelstraining-methods2402.17463Qwen / Alibaba CloudFeb 27, 2024score 9~121 min
- FebInternLM-Math: Open Math Large Language Models Toward Verifiable Reasoninguncategorized2402.06332InternLM / Shanghai AI LabFeb 9, 2024score 9~97 min
- JanSelf-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelsalignment2401.01335Jan 2, 2024score 9~128 min
- JanSecrets of RLHF in Large Language Models Part II: Reward Modelingalignment2401.06080Jan 11, 2024score 9~130 min
- JanSelf-Rewarding Language Modelsalignment2401.10020Jan 18, 2024score 9~116 min
- JanSliceGPT: Compress Large Language Models by Deleting Rows and Columnsarchitecture2401.15024Jan 26, 2024score 8~122 min
- JanRephrasing the Web: A Recipe for Compute and Data-Efficient Language Modelingdata2401.16380AppleJan 29, 2024score 9~109 min
- JanMedusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headsinference-optimization2401.10774Jan 19, 2024score 10~108 min
- JanSoaring from 4K to 400K: Extending LLM's Context with Activation Beaconllm-systems2401.03462Jan 7, 2024score 9~109 min
- JanLearning Universal Predictorspretraining2401.14953DeepMindJan 26, 2024score 7~105 min
- JanDeepSeek LLM: Scaling Open-Source Language Models with Longtermismscaling-laws2401.02954Jan 5, 2024~129 min
- JanLightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Modelstraining-methods2401.04658Jan 9, 2024score 9~118 min
- JanTuning Language Models by Proxytraining-methods2401.08565Jan 16, 2024~104 min
- JanAsynchronous Local-SGD Training for Language Modelingtraining-methods2401.09135DeepMindJan 17, 2024score 7~125 min
- JanWARM: On the Benefits of Weight Averaged Reward Modelstraining-methods2401.12187Jan 22, 2024score 9~112 min
2023
82- DecReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agentagents2312.10003Dec 15, 2023score 9~107 min
- DecNash Learning from Human Feedbackalignment2312.00886Dec 1, 2023score 9~110 min
- DecThe Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningalignment2312.01552Dec 4, 2023score 9~116 min
- DecAlignment for Honestyalignment2312.07000Dec 12, 2023score 9~124 min
- DecWeak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisionalignment2312.09390Dec 14, 2023score 9~115 min
- DecMagicoder: Source Code Is All You Needcode2312.02120Dec 4, 2023score 9~82 min
- DecLearning from One Continuous Video Streamtraining-methods2312.00598DeepMindDec 1, 2023~117 min
- DecBeyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelstraining-methods2312.06585Dec 11, 2023score 10~116 min
- NovRethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformersarchitecture2311.10642Nov 17, 2023score 9~106 min
- NovUltra-Long Sequence Distributed Transformerdistributed-training2311.02382Nov 4, 2023score 9~130 min
- NovZero Bubble Pipeline Parallelismdistributed-training2401.10241Nov 30, 2023score 9~105 min
- NovFlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Coresinference-optimization2311.05908Together AINov 10, 2023score 9~128 min
- NovQwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelsmultimodal2311.07919Qwen / Alibaba CloudNov 14, 2023score 8~116 min
- NovOrca 2: Teaching Small Language Models How to Reasonreasoning2311.11045Microsoft ResearchNov 18, 2023score 9~101 min
- NovAMSP: Super-Scaling LLM Training via Advanced Model States Partitioningtraining-methods2311.00257Nov 1, 2023score 9~117 min
- NovParameter-Efficient Orthogonal Finetuning via Butterfly Factorizationtraining-methods2311.06243Nov 10, 2023score 9~102 min
- NovDiLoCo: Distributed Low-Communication Training of Language Modelstraining-methods2311.08105DeepMindNov 14, 2023score 9~128 min
- NovFine-tuning Language Models for Factualitytraining-methods2311.08401Nov 14, 2023score 9~104 min
- NovCamels in a Changing Climate: Enhancing LM Adaptation with Tulu 2training-methods2311.10702Allen Institute for AINov 17, 2023score 10~106 min
- NovReplay across Experiments: A Natural Extension of Off-Policy RLtraining-methods2311.15951DeepMindNov 27, 2023~117 min
- OctSafe RLHF: Safe Reinforcement Learning from Human Feedbackalignment2310.12773Oct 19, 2023score 9~114 min
- OctMicroscaling Data Formats for Deep Learninglow-precision2310.10537Oct 16, 2023score 9~124 min
- OctImproved Baselines with Visual Instruction Tuningmultimodal2310.03744Stability AIOct 5, 2023score 9~103 min
- OctDemocratizing Reasoning Ability: Tailored Learning from Large Language Modelreasoning2310.13332Oct 20, 2023score 9~110 min
- OctContrastive Prefence Learning: Learning from Human Feedback without RLrl-training2310.13639Oct 20, 2023score 9~109 min
- OctCorrelated Noise Provably Beats Independent Noise for Differentially Private Learningtraining-methods2310.06771DeepMindOct 10, 2023~97 min
- OctLoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelstraining-methods2310.08659Oct 12, 2023score 9~110 min
- OctVeRA: Vector-based Random Matrix Adaptationtraining-methods2310.11454Oct 17, 2023score 9~107 min
- OctFP8-LM: Training FP8 Large Language Modelstraining-methods2310.18313Oct 27, 2023~108 min
- SepStabilizing RLHF through Advantage Model and Selective Rehearsalalignment2309.10202Sep 18, 2023score 9~119 min
- SepLarge Language Models for Compiler Optimizationcode2309.07062Sep 11, 2023~112 min
- SepDeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelsdistributed-training2309.14509Sep 25, 2023score 8~106 min
- SepAn Empirical Study of Scaling Instruct-Tuned Large Multimodal Modelsmultimodal2309.09958Sep 18, 2023score 9~101 min
- SepScaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuningpretraining2309.02591Sep 5, 2023score 10~113 min
- SepScaling Catalog Attribute Extraction with Multi-modal LLMsprompting2309.03409InstacartSep 7, 2023~116 min
- SepStatistical Rejection Sampling Improves Preference Optimizationrl-training2309.06657Sep 13, 2023score 9~114 min
- SepLMSYS-CHAT-1M: A LARGE-SCALE REAL-WORLD LLM CONVERSATION DATASETsafety2309.11998Sep 21, 2023~126 min
- SepEfficient RLHF: Reducing the Memory Usage of PPOtraining-methods2309.00754Sep 1, 2023score 9~102 min
- SepLongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelstraining-methods2309.12307Sep 21, 2023score 9~93 min
- SepEffective Long-Context Scaling of Foundation Modelstraining-methods2309.16039Sep 27, 2023score 9~104 min
- AugSelf-Alignment with Instruction Backtranslationalignment2308.06259Aug 11, 2023score 9~109 min
- AugDeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalesllm-systems2308.01320Aug 2, 2023score 9~96 min
- AugScaling Relationship on Learning Mathematical Reasoning with Large Language Modelsscaling-laws2308.01825Aug 3, 2023score 9~111 min
- AugStudying Large Language Model Generalization with Influence Functionstraining-methods2308.03296Aug 7, 2023score 8~102 min
- AugContinual Pre-Training of Large Language Models: How to (re)warm your model?training-methods2308.04014EleutherAIAug 8, 2023~112 min
- AugYARN: EFFICIENT CONTEXT WINDOW EXTENSION OF LARGE LANGUAGE MODELStraining-methods2309.00071Qwen / Alibaba CloudAug 31, 2023~112 min
- JulLlama 2: Open Foundation and Fine-Tuned Chat Modelsalignment2307.09288NVIDIAJul 18, 2023~120 min
- JulRetentive Network: A Successor to Transformer for Large Language Modelsarchitecture2307.08621Jul 17, 2023score 9~130 min
- JulFlashAttention-2: Faster Attention with Better Parallelism and Work Partitioninginference-optimization2307.08691Jul 17, 2023~110 min
- JulFocused Transformer: Contrastive Training for Context Scalingllm-systems2307.03170Jul 6, 2023score 9~116 min
- JulSecrets of RLHF in Large Language Models: Part I: PPOrl-training2307.04964Jul 11, 2023~131 min
- JulSet Learning for Accurate and Calibrated Modelstraining-methods2307.02245DeepMindJul 5, 2023~111 min
- JulRead Moretraining-methods2307.05695EleutherAIJul 11, 2023score 6~112 min
- JunPreference Ranking Optimization for Human Alignmentalignment2306.17492Jun 30, 2023score 9~118 min
- JunFaster Causal Attention Over Large Sequences Through Sparse Flash Attentionarchitecture2306.01160Jun 1, 2023score 9~101 min
- JunQuantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingpretraining2306.12929Jun 22, 2023score 8~107 min
- JunOrca: Progressive Learning from Complex Explanation Traces of GPT-4reasoning2306.02707Stability AIJun 5, 2023score 9~91 min
- JunFull Parameter Fine-tuning for Large Language Models with Limited Resourcestraining-methods2306.09782Jun 16, 2023score 9~100 min
- JunTraining Transformers with 4-bit Integerstraining-methods2306.11987Jun 21, 2023score 9~99 min
- JunOn-Policy Distillation of Language Models: Learning from Self-Generated Mistakestraining-methods2306.13649DeepMindJun 23, 2023score 10~116 min
- MaySLiC-HF: Sequence Likelihood Calibration with Human Feedbackalignment2305.10425May 17, 2023score 9~108 min
- MayLIMA: Less Is More for Alignmentalignment2305.11206May 18, 2023~96 min
- MayRead Morearchitecture2305.13048EleutherAIMay 22, 2023score 9~106 min
- MayGQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointsarchitecture2305.13245May 22, 2023~87 min
- MayBrainformers: Trading Simplicity for Efficiencyarchitecture2306.00008May 29, 2023score 9~88 min
- MayEnhancing Chat Language Models by Scaling High-quality Instructional Conversationsdata2305.14233OpenBMBMay 23, 2023score 9~99 min
- MayInstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningmultimodal2305.06500SalesforceMay 11, 2023score 9~101 min
- MayLet's Verify Step by Stepreasoning2305.20050May 31, 2023~124 min
- MayQLoRA: Efficient Finetuning of Quantized LLMstraining-methods2305.14314AI21May 23, 2023score 9~103 min
- MayOptimal Preconditioning and Fisher Adaptive Langevin Samplingtraining-methods2305.14442DeepMindMay 23, 2023~122 min
- MayThe False Promise of Imitating Proprietary LLMstraining-methods2305.15717May 25, 2023score 9~122 min
- MayFine-Tuning Language Models with Just Forward Passestraining-methods2305.17333May 27, 2023score 9~112 min
- MayDirect Preference Optimization: Your Language Model is Secretly a Reward Modeltraining-methods2305.18290Stability AIMay 29, 2023~122 min
- MayBlockwise Parallel Transformer for Long Context Large Modelstraining-methods2305.19370May 30, 2023score 8~112 min
- AprPyTorch FSDP: Experiences on Scaling Fully Sharded Data Paralleldistributed-training2304.11277Apr 21, 2023~121 min
- AprInstruction Tuning with GPT-4training-methods2304.03277Allen Institute for AIApr 6, 2023~113 min
- AprPyTorch FSDP: Experiences on Scaling Fully Sharded Data Paralleltraining-methods2304.11277Apr 21, 2023~121 min
- MarLLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentiontraining-methods2303.16199Mar 28, 2023~100 min
- FebToolformer: Language Models Can Teach Themselves to Use Toolstraining-methods2302.04761Feb 9, 2023~129 min
- FebPoisoning Web-Scale Training Datasets is Practicaltraining-methods2302.10149Feb 20, 2023~129 min
- JanOvercoming Simplicity Bias in Deep Networks using a Feature Sievetraining-methods2301.13293Jan 30, 2023~132 min
- JanThe Flan Collection: Designing Data and Methods for Effective Instruction Tuningtraining-methods2301.13688Allen Institute for AIJan 31, 2023~111 min
2022
15- DecConstitutional AI: Harmlessness from AI Feedbackalignment2212.08073Dec 15, 2022~123 min
- OctScaling Instruction-Finetuned Language Modelsalignment2210.11416SalesforceOct 20, 2022~105 min
- OctRobust Preference Learning for Storytelling via Contrastive Reinforcement Learningtraining-methods2210.07792EleutherAIOct 14, 2022~121 min
- SepAudioLM: a Language Modeling Approach to Audio Generationpretraining2209.03143Sep 7, 2022~118 min
- SepFlow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowtraining-methods2209.03003Sep 7, 2022~114 min
- SepIn-context Learning and Induction Headsuncategorized2209.11895Sep 24, 2022~120 min
- MayFew-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningalignment2205.05638May 11, 2022~115 min
- MayREAD MOREinference-optimization2205.14135Together AIMay 27, 2022~127 min
- MayUL2: Unifying Language Learning Paradigmspretraining2205.05131Together AIMay 10, 2022~122 min
- AprTraining a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackalignment2204.05862Apr 12, 2022~132 min
- AprStructured Pruning Learns Compact and Accurate Modelstraining-methods2204.00408Apr 1, 2022~113 min
- MarTraining language models to follow instructions with human feedbackalignment2203.02155Berkeley BAIRMar 4, 2022~125 min
- MarSTaR: Bootstrapping Reasoning With Reasoningreasoning2203.14465Mar 28, 2022~94 min
- JanAlpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learningdistributed-training2201.12023Jan 28, 2022~97 min
- JanGrokking: Generalization Beyond Overfitting on Small Algorithmic Datasetstraining-methods2201.02177Jan 6, 2022~125 min
2021
21- NovDeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingpretraining2111.09543Microsoft ResearchNov 18, 2021~106 min
- OctColossal-AI: A Unified Deep Learning System For Large-Scale Parallel Trainingdistributed-training2110.14883Oct 28, 2021~114 min
- OctMultitask Prompted Training Enables Zero-Shot Task Generalizationtraining-methods2110.08207EleutherAI561 citesOct 15, 2021~106 min
- SepFinetuned Language Models Are Zero-Shot Learnerstraining-methods2109.01652Sep 3, 2021~98 min
- SepRecursively Summarizing Books with Human Feedbacktraining-methods2109.1086266 citesSep 22, 2021~99 min
- AugTrain Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationpretraining2108.12409BaichuanAug 27, 2021~104 min
- JulSPLADE: Sparse Lexical and Expansion Model for First Stage Ranking and Re-rankingretrieval2107.05720Jul 12, 2021~95 min
- JunDecision Transformer: Reinforcement Learning via Sequence Modelingrl-training2106.01345Jun 2, 2021~111 min
- JunLoRA: Low-Rank Adaptation of Large Language Modelstraining-methods2106.09685Jun 17, 2021~91 min
- JunBitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelstraining-methods2106.10199Jun 18, 2021~102 min
- MayFNet: Mixing Tokens with Fourier Transformsarchitecture2105.03824Google ResearchMay 9, 2021~126 min
- MayGSPMD: General and Scalable Parallelization for ML Computation Graphsdistributed-training2105.04663May 10, 2021~138 min
- MayContrastive Learning for Many-to-many Multilingual Neural Machine Translationtraining-methods2105.09501May 20, 2021~117 min
- AprEfficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMdistributed-training2104.04473Apr 9, 2021~101 min
- AprZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningdistributed-training2104.07857Apr 16, 2021~116 min
- AprPrompt Tuning: The Power of Scale for Parameter-Efficient Prompt Tuningprompting2104.0869194 citesApr 18, 2021~88 min
- AprEmerging Properties in Self-Supervised Vision Transformersvision2104.14294Meta AI / FAIRApr 29, 2021~128 min
- FebTeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Modelsdistributed-training2102.07988Feb 16, 2021~109 min
- JanSwitch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitymoe2101.03961361 citesJan 11, 2021~109 min
- JanPrefix-Tuning: Optimizing Continuous Prompts for Generationtraining-methods2101.00190Jan 1, 2021~85 min
- JanZeRO-Offload: Democratizing Billion-Scale Model Trainingtraining-methods2101.06840Jan 18, 2021~102 min
2020
14- DecBinaryBERT: Pushing the Limit of BERT Quantizationlow-precision2012.15701Dec 31, 2020~104 min
- DecMemorizing Transformerstraining-methods2012.0036342 citesDec 1, 2020~111 min
- DecMaking Pre-trained Language Models Better Few-shot Learnerstraining-methods2012.15723Dec 31, 2020~117 min
- DecTraining data-efficient image transformers & distillation through attentionvision2012.12877Meta AI / FAIRDec 23, 2020~106 min
- OctFastFormers: Highly Efficient Transformer Models for Natural Language Understandinginference-optimization2010.133825 citesOct 26, 2020~122 min
- OctNeural Representations in Hybrid Recommender Systems: Prediction versus Regularizationtraining-methods2010.06070Oct 12, 2020~117 min
- SepLearning to summarize from human feedbackrl-training2009.01325Sep 2, 2020~94 min
- JulANCE: Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalretrieval2007.00808Jul 1, 2020~118 min
- JunPyTorch Distributed: Experiences on Accelerating Data Parallel Trainingdistributed-training2006.15704Jun 28, 2020~109 min
- JunConservative Q-Learning for Offline Reinforcement Learningrl-training2006.0477935 citesJun 8, 2020~107 min
- JunDeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameterstraining-methods2006.05525Jun 9, 2020~126 min
- FebBERT-of-Theseus: Compressing BERT by Progressive Module Replacingtraining-methods2002.02925Feb 7, 2020~113 min
- FebA Simple Framework for Contrastive Learning of Visual Representationsvision2002.05709Microsoft Research7,333 citesFeb 13, 2020~114 min
- JanReformer: The Efficient Transformerarchitecture2001.04451323 citesJan 13, 2020~104 min
2019
19- DecPyTorch: An Imperative Style, High-Performance Deep Learning Libraryarchitecture1912.01703Dec 3, 2019~105 min
- DecPEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationpretraining1912.08777Google ResearchDec 18, 2019~84 min
- DecDeep Double Descent: Where Bigger Models and More Data Hurtscaling-laws1912.02292149 citesDec 4, 2019~131 min
- NovMomentum Contrast for Unsupervised Visual Representation Learningvision1911.05722Nov 13, 2019~111 min
- OctZeRO: Memory Optimizations Toward Training Trillion Parameter Modelsdistributed-training1910.02054Stability AIOct 4, 2019~105 min
- OctQ8BERT: Quantized 8Bit BERTinference-optimization1910.06188Oct 14, 2019~91 min
- OctLudwig v0.3 Introduces Hyperparameter Optimization, Transformers and TensorFlow 2 supportpretraining1910.01108UberOct 2, 2019~108 min
- OctLearning Classifiers on Positive and Unlabeled Data with Policy Gradienttraining-methods1910.06535Oct 15, 2019~118 min
- OctQuantifying the Carbon Emissions of Machine Learningtraining-methods1910.09700Meta AI / FAIROct 21, 2019~87 min
- OctExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformertraining-methods1910.10683Allen Institute for AIOct 23, 2019~132 min
- OctBART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensiontraining-methods1910.13461Meta AI / FAIROct 29, 2019~102 min
- SepControlling Text Generation with Plug and Play Language Modelsrl-training1909.08593UberSep 18, 2019~120 min
- SepTinyBERT: Distilling BERT for Natural Language Understandingtraining-methods1909.10351Sep 23, 2019~106 min
- JulSpanBERT: Improving Pre-training by Representing and Predicting Spanspretraining1907.10529179 citesJul 24, 2019~119 min
- JulRoBERTa: A Robustly Optimized BERT Pretraining Approachpretraining1907.11692Meta AI / FAIRJul 26, 2019~113 min
- AprGenerating Long Sequences with Sparse Transformersarchitecture1904.10509Apr 23, 2019~125 min
- AprERNIE: Enhanced Representation through Knowledge Integrationpretraining1904.09223770 citesApr 19, 2019~100 min
- FebParameter-Efficient Transfer Learning for NLPtraining-methods1902.00751144 citesFeb 2, 2019~115 min
- FebLanguage Models are Unsupervised Multitask Learnerstraining-methods1902.01313Feb 4, 2019~107 min
2018
16- DecOff-Policy Deep Reinforcement Learning without Explorationtraining-methods1812.02900Dec 7, 2018~110 min
- NovGPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelismdistributed-training1811.06965236 citesNov 16, 2018~120 min
- OctA Block Coordinate Descent Proximal Method for Simultaneous Filtering and Parameter Estimationtraining-methods1810.06759Oct 16, 2018~93 min
- SepLarge Scale GAN Training for High Fidelity Natural Image Synthesispretraining1809.110961,786 citesSep 28, 2018~132 min
- JulBeyond Data and Model Parallelism for Deep Neural Networksdistributed-training1807.05358Jul 14, 2018~114 min
- JunDARTS: Differentiable Architecture Searcharchitecture1806.090551,402 citesJun 24, 2018~133 min
- JunNeural Ordinary Differential Equationsdiffusion1806.07366Jun 19, 2018~115 min
- JunPipeDream: Fast and Efficient Pipeline Parallel DNN Trainingdistributed-training1806.03377Jun 8, 2018~103 min
- JunQuantizing deep convolutional networks for efficient inference: A whitepaperlow-precision1806.08342Jun 21, 2018~102 min
- MarYear in Review: 2019 Highlights from the Uber Engineering Blogtraining-methods1803.03635UberMar 9, 2018~110 min
- MarGroup Normalizationuncategorized1803.08494478 citesMar 22, 2018~84 min
- FebSpectral Normalization for Generative Adversarial Networksarchitecture1802.05957Feb 16, 2018~122 min
- FebHorovod: Fast and Easy Distributed Deep Learning in TensorFlowdistributed-training1802.05799Feb 15, 2018~105 min
- FebAddressing Function Approximation Error in Actor-Critic Methodsrl-training1802.09477Feb 26, 2018~111 min
- JanSoft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorrl-training1801.01290Jan 4, 2018~116 min
- JanImproving Language Understanding by Generative Pre-Trainingtraining-methods1801.06146Jan 18, 2018~117 min
2017
14- DecQuantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencelow-precision1712.05877200 citesDec 15, 2017~113 min
- DecA general reinforcement learning algorithm that masters chess, shogi, and Go through self-playrl-training1712.01815Dec 5, 2017~122 min
- NovDecoupled Weight Decay Regularizationtraining-methods1711.05101Nov 14, 2017~107 min
- OctMixed Precision Trainingtraining-methods1710.03740203 citesOct 10, 2017~113 min
- Octmixup: Beyond Empirical Risk Minimizationtraining-methods1710.09412Oct 25, 2017~113 min
- AugFocal Loss for Dense Object Detectionvision1708.02002Aug 7, 2017~118 min
- JulFiber: Distributed Computing for AI Made Simplerl-training1707.06347UberJul 20, 2017~130 min
- MarPrototypical Networks for Few-shot Learningpretraining1703.0517585 citesMar 15, 2017~113 min
- MarReading Wikipedia to Answer Open-Domain Questionsretrieval1704.00051Mar 31, 2017~99 min
- MarMeta-Graph: Few-Shot Link Prediction Using Meta-Learningtraining-methods1703.03400Uber5,791 citesMar 9, 2017~118 min
- MarSurvey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationtraining-methods1703.09902Mar 29, 2017~142 min
- MarUnpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networksvision1703.10593Mar 30, 2017~112 min
- JanDeepETA: How Uber Predicts Arrival Times Using Deep Learningmoe1701.06538Uber268 citesJan 23, 2017~112 min
- JanWasserstein GANtraining-methods1701.07875Jan 26, 2017~99 min
2016
6- DecAttention Transfervision1612.03928Dec 12, 2016~110 min
- JulLayer Normalizationtraining-methods1607.06450Stability AIJul 21, 2016~98 min
- MarTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemsdistributed-training1603.04467Mar 14, 2016~107 min
- FebCommunication-Efficient Learning of Deep Networks from Decentralized Datallm-systems1602.056295,178 citesFeb 17, 2016~109 min
- FebBinarized Neural Networkslow-precision1602.02830Feb 9, 2016~111 min
- FebAsynchronous Methods for Deep Reinforcement Learningrl-training1602.01783Feb 4, 2016~138 min
2015
17- DecMXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systemsarchitecture1512.01274Dec 3, 2015~114 min
- DecDeep Residual Learning for Image Recognitionvision1512.03385AppleDec 10, 2015~106 min
- NovUnsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkspretraining1511.06434Stitch FixNov 19, 2015~111 min
- NovPrioritized Experience Replayrl-training1511.05952Stitch Fix2,045 citesNov 18, 2015~101 min
- NovBinaryConnect: Training Deep Neural Networks with Binary Weights during Propagationstraining-methods1511.00363Nov 2, 2015~95 min
- OctDeep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codinginference-optimization1510.00149Oct 1, 2015~94 min
- AugEffective Approaches to Attention-based Neural Machine Translationarchitecture1508.04025Aug 17, 2015~128 min
- AugUnified Acceleration Method for Packing and Covering Problems via Diameter Reductiontraining-methods1508.02439Aug 10, 2015~113 min
- JunLearning both Weights and Connections for Efficient Neural Networkpretraining1506.02626Jun 8, 2015~107 min
- MayHighway Networksarchitecture1505.00387May 3, 2015~98 min
- MarLudwig v0.3 Introduces Hyperparameter Optimization, Transformers and TensorFlow 2 supporttraining-methods1503.02531UberMar 9, 2015~122 min
- MarFaceNet: A Unified Embedding for Face Recognition and Clusteringvision1503.0383211,139 citesMar 12, 2015~111 min
- FebBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftarchitecture1502.03167Dropbox24,373 citesFeb 11, 2015~113 min
- FebTrust Region Policy Optimizationrl-training1502.05477Feb 19, 2015~108 min
- FebDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationtraining-methods1502.01852Feb 6, 2015~99 min
- —Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shifttraining-methods~110 min
- —On the importance of initialization and momentum in deep learningtraining-methods~117 min
2014
16- DecAdam: A Method for Stochastic Optimizationtraining-methods1412.6980Dec 22, 2014~125 min
- DecExplaining and Harnessing Adversarial Examplesuncategorized1412.65728,153 citesDec 20, 2014~141 min
- NovHow transferable are features in deep neural networks?training-methods1411.1792Nov 6, 2014~123 min
- NovGloVe: Global Vectors for Word Representationtraining-methods1411.5595Nov 20, 2014~102 min
- OctNeural Turing Machinesarchitecture1410.5401110 citesOct 20, 2014~129 min
- OctHigh-Performance Distributed ML at Scale through Parameter Server Consistency Modelstraining-methods1410.8043Oct 29, 2014~108 min
- SepNeural Machine Translation by Jointly Learning to Align and Translatealignment1409.0473Sep 1, 2014~101 min
- SepOn the Properties of Neural Machine Translation: Encoder–Decoder Approachesscaling-laws1409.1259Sep 3, 2014~114 min
- AugConvolutional Neural Networks for Sentence Classificationarchitecture1408.5882Aug 25, 2014~108 min
- JunLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translationarchitecture1406.1078Jun 3, 2014~119 min
- JunGenerative Adversarial Networksarchitecture1406.2661Google ResearchJun 10, 2014~117 min
- MarMeta-Graph: Few-Shot Link Prediction Using Meta-Learningpretraining1403.6652UberMar 26, 2014~118 min
- JanNeural Word Embedding as Implicit Matrix Factorizationno summary yetpretrainingext:word-embed-mf-2014Jan 1, 2014
- —Generative Adversarial Netstraining-methods~16 min
- —Dropout: A Simple Way to Prevent Neural Networks from Overfittingtraining-methods~21 min
- —Dropout: A Simple Way to Prevent Neural Networks from Overfittingtraining-methods~114 min
2013
4- DecAuto-Encoding Variational Bayesarchitecture1312.6114Stitch FixDec 20, 2013~109 min
- DecExact solutions to the nonlinear dynamics of learning in deep linear neural networkstraining-methods1312.6120Dec 20, 2013~119 min
- OctDistributed Representations of Words and Phrases and their Compositionalitypretraining1310.454618,086 citesOct 16, 2013~114 min
- FebMaxout Networksarchitecture1302.4389Feb 18, 2013~131 min
2012
22009
12000
19- —A decision-theoretic generalization of on-line learning and an application to boostingtraining-methods~84 min
- —Adaptive Subgradient Methods for Online Learning and Stochastic Optimizationtraining-methods~136 min
- —Large Scale Distributed Deep Networkstraining-methods~103 min
- —Regularization and Variable Selection via the Elastic Nettraining-methods~118 min
- —Understanding the difficulty of training deep feedforward neural networkstraining-methods~115 min
- —Adapting Large Language Models for Document-Level Machine Translationtraining-methods~110 min
- —Hogwild!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descenttraining-methods~113 min
- —Least Angle Regressiontraining-methods~121 min
- —Map-Reduce for Machine Learning on Multicoretraining-methods~141 min
- —Regularized Multi-Task Learningtraining-methods~117 min
- —Netflix Update: Try This at Hometraining-methods~112 min
- —Online Convex Programming and Generalized Infinitesimal Gradient Ascenttraining-methods~99 min
- —Some PAC-Bayesian Theoremstraining-methods~110 min
- —Pegasos: Primal Estimated sub-GrAdient SOlver for SVMtraining-methods~99 min
- —Rainbow: Combining Improvements in Deep Reinforcement Learningtraining-methods~127 min
- —Sequence to Sequence Learning with Neural Networkstraining-methods~109 min
- —Large-Scale Machine Learning with Stochastic Gradient Descenttraining-methods~109 min
- —Support Vector Machine Learning for Interdependent and Structured Output Spacestraining-methods~121 min
- —Distributed Representations of Words and Phrases and their Compositionalitytraining-methods~123 min