Davis Treybig
Currently reading

A running list of papers, blogs, and technical writing I've found interesting across infrastructure, AI, and systems.

01Challenges in Scaling RL (Periodic Labs)Presentation by RL lead at Periodic Labs about challenges in scaling RL. Highlights three key ideas. First, all RL is inherently partial off policy because the inference engines (SGLang, VLLM) are different from the training engines (Megatron), leading to implicit drift. Notable examples of this are kernel differences, batch size differences, and routers in MoE models. They do some interesting work to address this like replay the router output in training, NOT re-run the router. Second, async RL is required to keep GPU utilization high, but this adds system complexity. Third, async RL will be definitionally off policy which requires thoughtful consideration of what to do when the rollout is too far from the active policy. They try to mask cases where this is very extreme↗02Scaling Synthetic RL EnvironmentsBlog post on the challenges and opportunities around scaling synthetic RL environments. Good discussion of the complexity that emerges when you start wanting to treat the environment layer, the runtime layer, and the training layer as different systems that interact in complex ways. Particularly interesting points include: 1. Separation of control plane, data plane, verification plane, and training plane, 2. Building modular building blocks for environment generation, 3. How the state and trajectory accumulated during a trace is actually more computationally intensive than spinning up the environment in the first place, and thus the criticality of snapshotting and being able to rewind/reconstruct trajectories, and 4. The challenges of async RL in terms of identifying whether rollouts are worth including vs. not and how to manage more off-policy rollouts↗03Karpathy at Sequoia AI Ascent 2026Karpathy review of recent changes in AI at Sequoia 2026 AI Ascent↗04Diffusion and Autoregressive Language Models: Two Ends of a SpectrumVideo discussing the parallels between diffusion language models and autoregressive language models. Essentially, diffusion and AR are more like different ends of a spectrum than binary choices. The talk ends with an interesting discussion of learned order models where instead of a fixed left-to-right generation order (AR) or a fixed random generation order (diffusion) you learn a policy for where to insert each subsequent token and then predict what the value of that token should be↗05Hardware-Aware Dynamic Speculative DecodingBlog post from Cohere about hardware-aware speculative decoding. Speculative decoding uses a small draft model to propose multiple tokens at a time which the AR model then accepts/rejects/edits. This can speed up AR models, but it doesn't work in all situations. In particular, at large batch sizes, you have already moved towards a more compute bound regime, thus you don't have the excess compute to run the SD model. In contrast, in low batch sizes, you are memory bound so you have a lot of excess "free" compute you can use for the SD model. Cohere proposes a dynamic SD algorithm that considers this and can dynamically run an SD model based on hardware + batch size↗06Serving DeepSeek-V4: Why Million-Token Context Is an Inference Systems ProblemTogether.ai blog on serving DeepSeek 4. Good discussion of how DeepSeek explored a different attention architecture that has some layers look only at recent tokens, some layers look at a compressed version of the entire context, and some do a top-k lookup of a compressed version of different chunks of the context. Notably, DeepSeek also actually store each of these as completely different KV caches, never storing the baseline raw KV cache. This complicates serving a lot as it heavily alters the way serving works, but it gives a big efficiency gain↗07Adaptive Parallel ReasoningPaper exploring embedding parallel reasoning directly inside of an LLM via combining special token generation with an altered inference engine. Cool idea vs. doing the parallel reasoning more at the orchestration layer like you see in most papers exploring best-of-N or majority voting or similar ideas↗08Self-Distillation Enables Continual LearningSelf distillation is an increasingly popular technique for fine tuning models. The general problem statement is that supervised fine tuning is easy to scale data for but is off-policy, whereas RL is harder to scale data for but is on-policy. On-policy methods tend to be better learning methods. The key insight is that you take a base model, sample an output, and then re-run that base model on that task with additional context such as a demonstrated example of solving a similar problem. The "teacher" model here with additional context should be able to do a much better job of answering the question, and then what you optimize is the delta in distributions between the two models. But, because the "teacher" is just the student with more context, it is an on policy method↗09Scaling Managed Agents: Decoupling the Brain from the HandsBlog by Anthropic on the right abstractions for agents. I like their framing of the core, separated components being the harness, the sandbox, the session log, the orchestrator, and tools. This has strong benefits vs. the fully coupled, everything in a container approach↗10Automated Design of Agentic SystemsOpen endedness paper on automating the process of discovering optimal agents via a meta-search process where one meta agent writes new agents and then keeps track of interesting ones to keep writing better agents↗11Darwin Gödel Machine: Open-Ended Evolution of Self-Improving AgentsGood openendedness paper discussing a self-modifying coding agent that via evolutionary search improves its own harness to dramatically improve coding performance↗12End-to-End Test-Time Training for Long ContextPaper that explores test-time training by updating model weights based on the context you want to retrieve against. The core intuition is that this can be a type of learned "attention" that is more compressed than running full attention on every token generation. Similar to Google's TITANS work.↗13MIRASResearch blog by Google that describes a better formalism for sequence modeling and memory in foundation models. Thus far, there have been a lot of tweaks on full-attention (Mamba, SSMs, etc) and then of course RNNs can be viewed as a different type of context/memory compression, but there is not a strong baseline of all possible algorithms or methods for considering past sequence data in next-token prediction. As part of the blog, they also discuss TITAN, a methodology where you have a learned memory structure that can be attended to during inference time and which is continuously updated based on "surprise" the model sees. Their results are really interesting especially for very long contexts.↗14On The Landscape of Spoken Language Models: A Comprehensive SurveyAmazing overview of the current state of spoken language models. Basically, highlights how we are in a similar spot NLP was a few years ago, with a ton of single-function models like sentence classifiers, but no universal generalist model. But like language, we should eventually get to a generalist speech model.↗15CWM: An Open-Weights LLM for Research on Code Generation with World ModelsA code gen "world model" from Facebook that predicts changes in the environment like environment variables at the same time as it suggests the next token. Very cool idea.↗16When AI Writes the World's Software, Who Verifies It?Great overview in the recent progress in using AI coding models to autoformalize software systems in Lean↗17Quantization for On-Device ModelsReally impressive results on quantizing models to run on Apple devices. Mirai pairs a custom quantization stack with a specialized inference engine, using post-training quantization (a modified YAQA algorithm) followed by quantization-aware distillation, plus Random Hadamard Transforms to suppress outliers. The standout result is that their 4-bit models deliver 40-60% more tokens per second at the same quality as tools like llama.cpp, and their 8-bit versions run faster than llama.cpp's 4-bit models while staying nearly identical to full precision. The key insight is hardware-aware design - optimizing for GPU efficiency rather than pure compression ratios.↗18Learning to Discover at Test TimePaper exploring methods for doing reinforcement learning at test-time for the purposes of discovery. It makes the interesting point that discovery problems look very different from standard learning objectives, because you care more about finding a single state-of-the-art method than the average performance of the LLM. This lets you alter many fundamental assumptions - such as being OK with updating model weights at test time for a given task, or changing the RL objective to maximize variance rather than optimize for average outcome. They show some really cool results applying this to GPU kernel writing.↗19Pre-training Isn't Bitter EnoughCMU research paper suggesting that the fact that we still hand-craft the learning tasks for language models is to an extent anti-bitter-lesson. While we have generalized learning algorithms, we don't have generalized task-optimization algorithms, and instead we basically hand-tune self-supervised learning objectives when training LLMs. The paper explores an alternative where you instead have a system that co-learns both the learning objective and the model parameters, under the intuition that a learning objective delta that drives a better gradient in the model learning is likely to be a better objective.↗20SciPredict: Can LLMs Predict the Outcomes of Scientific Experiments?Interesting paper that benchmarks how well LLMs can forecast the results of real scientific experiments across natural science domains.↗21Bayesian Forecasting with LLMsInteresting paper on methodologies to make LLMs better forecasters, treating the forecasting state as a buildup of bayesian style probabilities with evidence↗22CL-Bench 1.0New benchmark for continual learning, evaluating how well models retain and build on prior knowledge without catastrophic forgetting.↗23ProgramBenchFacebook Research benchmark for evaluating program synthesis and code understanding capabilities of LLMs.↗24Long-Running ClaudeCool blog by Anthropic about using a long running agent to implement a differentiable Boltzmann solver in JAX. Good example of how to structure a long running agent program in terms of instructions, tests, etc. I think right now this only works in these very, very verifiable domains - in this case there was a reference implementation to compare against.↗25Composer 2Overview of how Cursor trained Composer 2. Some particularly interesting discussions of their training mix (Kimi 2 base, continued pre-training, SFR, then RL), how they do RL (e.g. sequence parallelism, updating model weights mid rollout), and their environments infrastructure. I also like some of the discussion on building evals/benchmarks that are more representative of user behavior - e.g. focusing on under specified queries.↗26Uni 1New model from Luma that is a single, decoder only, autoregressive transformer that interleaves images and text on both input and output. As a result, it appears to have much stronger visual reasoning capabilities, and can also support much more complex input conditioning with long prompts & other image inputs. The controllability is particularly good.↗27After WIMPGood blog playing out how the fundamental assumptions on how web services and users interact is changing. Traditionally, the GUI/UI was the fixed exchange protocol between a user and a service. The future probably looks different - maybe some kind of UI/GUI scaffolding or principles, coupled with some kind of user-attached personal software libraries that outline preferences of that user, and dynamically compile into an application.↗28Challenges and Research Directions for Large Language Model Inference HardwareAmazing overview of how LLM inference merits a structural rethinking of chip & datacenter design. The insane growth of inference vs. training, coupled with novel architectures like MoE & long context, mean that the decode step of LLM inference is increasingly memory bound and latency constrained. They propose four major areas of research to address this - more of a focus on flash based memory vs. just DRAM/HBM, processing near memory (e.g. small compute units located closer to memory) for higher bandwidth, 3D memory stacking for higher bandwidth, and lower latency interconnect approaches.↗29UniFusion: Vision-Language Model as Unified Encoder in Image GenerationPaper from Adobe exploring using a VLM as the universal encoding layer for generative image models, replacing the more common separation of a vision + text encoder. Approach seems to result in models that generalize better and have better transfer in training, though it requires some nuance in how you apply VLM.↗30Interesting Directions in VisionGood overview of some recent trends in vision-language-action models for robotics. Main themes include: integrating tactile data, incorporating 3D reasoning and 3D priors, applying RL on top of base VLM/VLA, and unifying world models with VLAs↗31Agent Design Patterns OverviewIncredible overview of recent design patterns in agents. Resounding theme is adopting computing primitives as the basis set of tools (e.g. bash, file system, code) & offloading all context management to the computer↗32Recursive Language ModelsA proposed system architecture for models/agents, designed around recusively calling LLMs in a REPL environment where the context is represented as a variable in memory that is not shown in any way to the LLM unless it specifically asks for it using various tools (grep, peeking, etc). You essentially are asking the LLM to figure out how to probe & discern the context and identify how to manipulate it with a sequence of recursive sub-calls. In a way this is similar to how systems like Claude Code work, but they are coming more from the model design side vs. a task-specific solution. Amazing paper.↗33Towards a Science of Scaling Agent SystemsInteresting overview of how multi-agent vs. single-agent system design variations impact task quality. High variance of whether it improves or degrades in what situations↗34Automated Self-TestingGreat blog by Replit on agent design for automated testing of code gen agents. Particularly interesting was the design strategy of simply their computer use agent simply write playwright code rather than have specialized tools like select element, etc.↗35Sandbox RuntimeA lightweight agent sandboxing tool from Anthropic, based around filesystem and network restrictions using OS utilities vs. using full fledged container or microVM.↗36Everything is Context: Agentic File System Abstraction for Context EngineeringPaper exploring how the file system access can be an effective tool for context management. Similar to the Manus architecture.↗37Inside ThunderKittens' Python BindingsInteresting overview of part of the ThunderKittens, a framework for writing more performant GPU kernels out of Stanford↗38Radiance Fields and Future of Generative MediaGreat overview of the state of neural radiance fields and the role that 3D models will play in generative media.↗39Cheap RL Tasks Will Waste ComputeInteresting argument on why the world is shifting to extremely high-end, high cost, specific RL data away.↗40Principles of Diffusion ModelsHolistic overview of the principles of diffusion models.↗41Scaling Reasoning in Diffusion Large Language Models via Reinforcement LearningFirst example of applying RL to improve reasoning in diffusion models.↗42LLMs for Scheduling Policies in Distributed SystemsCool example of using an LLM + simulator to optimize a database scheduler. Generator + verifier pattern.↗43Barbarians at the GateInteresting overview of ideas for applying ML to systems research.↗44Supporting Our AI Overlords: Redesigning Data Systems to be Agent-FirstAgent-first data system design from Berkeley.↗45Improving Cursor Tab with Online RLOverview of how Cursor does online RL to improve their Cursor Tab model.↗46Denny Zhou – Reasoning SlidesGood slides on reasoning models and techniques.↗47SkyRL v0.1 — NovaSkyModular RL framework with separate trainer, environment, generator, and reward layers.↗48The Second Half of Machine LearningGood blog on how ML is moving from methods to environments and RL.↗49WeaverCombining multiple weakly supervised verifiers into a strong ensemble verifier.↗50BlockDiff Incremental VM SnapshotsCognition's OSS code sandbox designed for snapshotting.↗51TITANSAlternative autoregressive architecture with dynamic memory blocks for long context.↗52TAO: Test-Time Compute to Train Efficient LLMsCool example of using reasoning models to autonomously fine tune LLMs.↗53Scaling RL ComputeDiscussion of how to scale RL compute and its bottlenecks from General Reasoning.↗54Inductive Moment MatchingDiffusion-like method allowing discrete jumps in sampling and better use of pretrained networks.↗55Trellis3DUnified latent representation for generative 3D objects decoding to radiance fields, Gaussians, and meshes.↗56SIMATrains agents to act in diverse 3D game worlds from language inputs; notably it generalizes well across games where it has no game specific training.↗57Autoregressive Image Generation via Progressive UpsamplingTreats image generation as autoregressive refinement to higher resolutions, competitive with diffusion.↗58Flow MatchingGeneral method for training generative models via matching probability flows instead of noise corruption.↗59ChameleonMixed-modality model encoding text and images in a single token space trained end-to-end.↗60Training Verifiers for Math ProblemsEarly OpenAI paper on training verifiers for mathematical reasoning.↗61MuZeroModel-free RL algorithm mastering games without explicit rule knowledge.↗62Beam SearchUsing beam search as a test-time reasoning strategy.↗63Tree of ThoughtsReasoning strategy that generates candidate thoughts and explores them via tree search heuristics.↗64LLM ReasonersUnifies reasoning as reward technique, world model, and search algorithm; shows search and RAP outperform basic CoT.↗65Reasoning with Language Model is Planning with World ModelCombines a generator and a world-model LLM with MCTS for iterative reasoning.↗66Large Language MonkeysDemonstrates large gains from sampling many outputs and selecting via a verifier.↗67Beyond A*Trains transformers on A*-generated traces to internalize search-like problem solving.↗68Stream of SearchShows that post-training on search-style reasoning traces greatly improves CoT performance.↗69DualFormerTrains on full and partially masked reasoning traces to enable fast vs slow reasoning modes.↗70Training LLMs to Reason in a Continuous Latent SpacePerforms reasoning in latent space instead of over discrete token sequences.↗71Byte Latent TransformersTokenizer-free transformer operating on bytes with entropy-based dynamic patching.↗72The State of Generative Models - 2024 ReviewOverview of late-2024 trends in multimodality, reasoning, tokenization removal, and agents.↗73Model Context ProtocolAnthropic's framework for standardizing tools for LLMs.↗74FastMCPCool framework from Prefect that simplifies building production MCP servers↗75BrushWebGPU and Burn-based engine for training and rendering Gaussian splats.↗76UnboundedPrototype "infinite" game powered by distilled LLMs and diffusion models.↗77lolcatsConverts transformers into linear/state-space style models via attention replacement and LoRA.↗78SQLite in Durable ObjectsSynchronous embedded SQLite inside Cloudflare Durable Objects for session backends.↗79Differential TransformerInteresting idea of modulating attention to a relative score, instead of absolute, in theory reducing attentinon towards irrelevant context.↗80Networks of NetworksCool paper demonstrating how simple compound AI system designs (e.g. judge/verifier, best of K voting) can produce huge performance deltas.↗81MaestroNetflix's JSON-based workflow orchestrator supporting DAG and cyclic graphs.↗82Resource Management for Aurora ServerlessSome interesting discussion on resource and memory management in Aurora Serverless.↗83OpenHouseLinkedIn's open-source control plane/catalog for lakehouse architectures.↗84Exploiting Cloud Object Storage for High-Performance AnalyticsPaper exploring opitmal system design for querying cloud object stores efficiently.↗85The Architecture of Serverless Data SystemsAmazing six-part series on serverless data system design (Aurora, Dynamo, Neon, Kora, etc.).↗86Apple Intelligence OverviewOverview of Apple Intelligence system design & architecture.↗87Accelerating Code Migrations with AIGoogle deep dive on techniques for AI-assisted codebase migration.↗88Efficient finetuning of Llama 3 with FSDP QDoRAAnswer.ai blog an their "continued pre-training" method, QDoRA↗89Hybrid ML + Numerical Weather ModelCombines ML with traditional numerical weather prediction for long-range forecasts and uncertainty bounds.↗90Privacy in Public + Private RetrievalExploration of how to handle retrieval in AI systems assuming mix of private + public data to retrieve over.↗91Dynamic Partitioning for VisualizationTechniques for dynamic partitioning in data visualization.↗92Draco 2Renderer agnostic data visualization format designed to allow for flexible encoding of visualization rules.↗93DynaVisCool idea to dynamically synthesize data visualization editor widgets based on the data visualization task.↗94Formalizing Visualization Design Knowledge as Constraints: Actionable and Extensible Models in DracoFoundational paper on the Draco constraint-based visualization system.↗95SWE-AgentBenchmark for coding agents.↗96In Defense of Dual-Encoders for RerankingExplains why dual encoders underperform cross encoders and how to fix them.↗97Scaling MonosemanticitySeminal Anthropic blog about how to identify semantic features and their associated neurons in deep neural networks.↗98GPUs Go BrrrDiscusses GPU kernel optimization and hardware-aware AI system design.↗99GenieSuper interesting idea of inferring not just videos but action controllable worlds from input images. You train self-supervised on existing video data, and you learn both how to predict the next frame and the action that would have connected those two frames.↗100Foundation Models for Reasoning on ChartsCool applications of foundation models for reasoning about data visualization charts.↗101MeerkatExploration of a dataframe library that natively supports unstructured data types.↗102Reka Core / Edge Tech ReportTechnical report on Reka Core and Edge models.↗103GorillaX Exec EngineInteresting tool-use runtime that natively supports various ideas like undo.↗104RAFTFine tuning strategy to optimize for domain specific RAG workflows.↗105FASTERHigh-performance key-value store with an elegant log-structured design.↗106GarnetMicrosoft's high-performance KV store related to FASTER.↗107LIDASystem for AI-assisted data storytelling and visualization.↗108Mechanistic Design & Scaling of Hybrid ArchitecturesInteresting paper highlighting ways to mechanistically test models in small scale, specific tasks in ways that predict scaling properties, allowing for many architectural approaches to be tested rapidly↗109LumiereVideo generative model with improved temporal–spatial consistency.↗110Large Sequence Models for Software EngineeringCode models trained on the software engineering process (reviews, debugging, etc.).↗111Scaling Data-Constrained Language ModelsDiscusses strategies for scaling models when high-quality data is limited.↗