Currently reading

A running list of papers, blogs, and technical writing I've found interesting across infrastructure, AI, and systems.

01Dust: Pretraining Transformers Without Backpropagation (Q Labs)Basically all machine learning models are trained via gradient descent-based backpropagation. This paper explores a zeroeth order method that does not use backpropagation, in theory allowing for a non-differentiable model architecture (such as non-linear neuron layers, etc). The intuition for why to explore this is that - backpropagation is very efficient but is definitionally a type of model prior, prioritizing optimizing against the first order derivative over other methods of searching over the manifold of model weights. A naive example might be that following the first order derivative could lead to a local minima/maxima, whereas understanding the curvature of the domain you are in could make it easier to escape that. The argument is that as we enter more compute-rich domains, methods that take more of a brute force "search" approach may be superior, even if they are theoretically less efficient. These guys show a proof of concept of this by pretraining a language model via a method that randomly perturbs the outputs of every neuron, and estimate the local gradient around each neuron via such perturbations. You need to do a lot of clever things to get this to work - one key insight being perturbing the activations not the weights so you can get more examples in a single forward pass. They show a number of results where at a small scale, with sufficient compute, this can nearly match backpropagation which is quite interesting↗02Computational Depth Is All You Need (Q Labs)Amazing blog post on how under-explored computational depth is in neural networks. Most model axes have been scaled 1000x over the past few years (data, test time compute, parameters, sparsity, etc) with the sole exception being depth. CoT reasoning is the effective way most models achieve greater depth, but this is extremely inefficient given a single token can not come close to representing the expressive richness of the full set of neurons in a model. The Q Labs team identifies some interesting solutions to the challenges of pursuing greater computational depth (such as two phase training where you expand model depth later in training), and shows some extremely interesting small scale results. Spirit of the research similar to CoreAuto's recent model depth competition.↗03Challenges in Scaling RL (Periodic Labs)Presentation by RL lead at Periodic Labs about challenges in scaling RL. Highlights three key ideas. First, all RL is inherently partial off policy because the inference engines (SGLang, VLLM) are different from the training engines (Megatron), leading to implicit drift. Notable examples of this are kernel differences, batch size differences, and routers in MoE models. They do some interesting work to address this like replay the router output in training, NOT re-run the router. Second, async RL is required to keep GPU utilization high, but this adds system complexity. Third, async RL will be definitionally off policy which requires thoughtful consideration of what to do when the rollout is too far from the active policy. They try to mask cases where this is very extreme↗04Scaling Synthetic RL EnvironmentsBlog post on the challenges and opportunities around scaling synthetic RL environments. Good discussion of the complexity that emerges when you start wanting to treat the environment layer, the runtime layer, and the training layer as different systems that interact in complex ways. Particularly interesting points include: 1. Separation of control plane, data plane, verification plane, and training plane, 2. Building modular building blocks for environment generation, 3. How the state and trajectory accumulated during a trace is actually more computationally intensive than spinning up the environment in the first place, and thus the criticality of snapshotting and being able to rewind/reconstruct trajectories, and 4. The challenges of async RL in terms of identifying whether rollouts are worth including vs. not and how to manage more off-policy rollouts↗05Karpathy at Sequoia AI Ascent 2026Karpathy review of recent changes in AI at Sequoia 2026 AI Ascent↗06Diffusion and Autoregressive Language Models: Two Ends of a SpectrumVideo discussing the parallels between diffusion language models and autoregressive language models. Essentially, diffusion and AR are more like different ends of a spectrum than binary choices. The talk ends with an interesting discussion of learned order models where instead of a fixed left-to-right generation order (AR) or a fixed random generation order (diffusion) you learn a policy for where to insert each subsequent token and then predict what the value of that token should be↗07Hardware-Aware Dynamic Speculative DecodingBlog post from Cohere about hardware-aware speculative decoding. Speculative decoding uses a small draft model to propose multiple tokens at a time which the AR model then accepts/rejects/edits. This can speed up AR models, but it doesn't work in all situations. In particular, at large batch sizes, you have already moved towards a more compute bound regime, thus you don't have the excess compute to run the SD model. In contrast, in low batch sizes, you are memory bound so you have a lot of excess "free" compute you can use for the SD model. Cohere proposes a dynamic SD algorithm that considers this and can dynamically run an SD model based on hardware + batch size↗08Serving DeepSeek-V4: Why Million-Token Context Is an Inference Systems ProblemTogether.ai blog on serving DeepSeek 4. Good discussion of how DeepSeek explored a different attention architecture that has some layers look only at recent tokens, some layers look at a compressed version of the entire context, and some do a top-k lookup of a compressed version of different chunks of the context. Notably, DeepSeek also actually store each of these as completely different KV caches, never storing the baseline raw KV cache. This complicates serving a lot as it heavily alters the way serving works, but it gives a big efficiency gain↗09Adaptive Parallel ReasoningPaper exploring embedding parallel reasoning directly inside of an LLM via combining special token generation with an altered inference engine. Cool idea vs. doing the parallel reasoning more at the orchestration layer like you see in most papers exploring best-of-N or majority voting or similar ideas↗10Self-Distillation Enables Continual LearningSelf distillation is an increasingly popular technique for fine tuning models. The general problem statement is that supervised fine tuning is easy to scale data for but is off-policy, whereas RL is harder to scale data for but is on-policy. On-policy methods tend to be better learning methods. The key insight is that you take a base model, sample an output, and then re-run that base model on that task with additional context such as a demonstrated example of solving a similar problem. The "teacher" model here with additional context should be able to do a much better job of answering the question, and then what you optimize is the delta in distributions between the two models. But, because the "teacher" is just the student with more context, it is an on policy method↗11Scaling Managed Agents: Decoupling the Brain from the HandsBlog by Anthropic on the right abstractions for agents. I like their framing of the core, separated components being the harness, the sandbox, the session log, the orchestrator, and tools. This has strong benefits vs. the fully coupled, everything in a container approach↗12Automated Design of Agentic SystemsOpen endedness paper on automating the process of discovering optimal agents via a meta-search process where one meta agent writes new agents and then keeps track of interesting ones to keep writing better agents↗13Darwin Gödel Machine: Open-Ended Evolution of Self-Improving AgentsGood openendedness paper discussing a self-modifying coding agent that via evolutionary search improves its own harness to dramatically improve coding performance↗14End-to-End Test-Time Training for Long ContextPaper that explores test-time training by updating model weights based on the context you want to retrieve against. The core intuition is that this can be a type of learned "attention" that is more compressed than running full attention on every token generation. Similar to Google's TITANS work.↗15MIRASResearch blog by Google that describes a better formalism for sequence modeling and memory in foundation models. Thus far, there have been a lot of tweaks on full-attention (Mamba, SSMs, etc) and then of course RNNs can be viewed as a different type of context/memory compression, but there is not a strong baseline of all possible algorithms or methods for considering past sequence data in next-token prediction. As part of the blog, they also discuss TITAN, a methodology where you have a learned memory structure that can be attended to during inference time and which is continuously updated based on "surprise" the model sees. Their results are really interesting especially for very long contexts.↗16On The Landscape of Spoken Language Models: A Comprehensive SurveyAmazing overview of the current state of spoken language models. Basically, highlights how we are in a similar spot NLP was a few years ago, with a ton of single-function models like sentence classifiers, but no universal generalist model. But like language, we should eventually get to a generalist speech model.↗17CWM: An Open-Weights LLM for Research on Code Generation with World ModelsA code gen "world model" from Facebook that predicts changes in the environment like environment variables at the same time as it suggests the next token. Very cool idea.↗18When AI Writes the World's Software, Who Verifies It?Great overview in the recent progress in using AI coding models to autoformalize software systems in Lean↗19Quantization for On-Device ModelsReally impressive results on quantizing models to run on Apple devices. Mirai pairs a custom quantization stack with a specialized inference engine, using post-training quantization (a modified YAQA algorithm) followed by quantization-aware distillation, plus Random Hadamard Transforms to suppress outliers. The standout result is that their 4-bit models deliver 40-60% more tokens per second at the same quality as tools like llama.cpp, and their 8-bit versions run faster than llama.cpp's 4-bit models while staying nearly identical to full precision. The key insight is hardware-aware design - optimizing for GPU efficiency rather than pure compression ratios.↗20Learning to Discover at Test TimePaper exploring methods for doing reinforcement learning at test-time for the purposes of discovery. It makes the interesting point that discovery problems look very different from standard learning objectives, because you care more about finding a single state-of-the-art method than the average performance of the LLM. This lets you alter many fundamental assumptions - such as being OK with updating model weights at test time for a given task, or changing the RL objective to maximize variance rather than optimize for average outcome. They show some really cool results applying this to GPU kernel writing.↗21Pre-training Isn't Bitter EnoughCMU research paper suggesting that the fact that we still hand-craft the learning tasks for language models is to an extent anti-bitter-lesson. While we have generalized learning algorithms, we don't have generalized task-optimization algorithms, and instead we basically hand-tune self-supervised learning objectives when training LLMs. The paper explores an alternative where you instead have a system that co-learns both the learning objective and the model parameters, under the intuition that a learning objective delta that drives a better gradient in the model learning is likely to be a better objective.↗22SciPredict: Can LLMs Predict the Outcomes of Scientific Experiments?Interesting paper that benchmarks how well LLMs can forecast the results of real scientific experiments across natural science domains.↗23Bayesian Forecasting with LLMsInteresting paper on methodologies to make LLMs better forecasters, treating the forecasting state as a buildup of bayesian style probabilities with evidence↗24CL-Bench 1.0New benchmark for continual learning, evaluating how well models retain and build on prior knowledge without catastrophic forgetting.↗25ProgramBenchFacebook Research benchmark for evaluating program synthesis and code understanding capabilities of LLMs.↗26Long-Running ClaudeCool blog by Anthropic about using a long running agent to implement a differentiable Boltzmann solver in JAX. Good example of how to structure a long running agent program in terms of instructions, tests, etc. I think right now this only works in these very, very verifiable domains - in this case there was a reference implementation to compare against.↗27Composer 2Overview of how Cursor trained Composer 2. Some particularly interesting discussions of their training mix (Kimi 2 base, continued pre-training, SFR, then RL), how they do RL (e.g. sequence parallelism, updating model weights mid rollout), and their environments infrastructure. I also like some of the discussion on building evals/benchmarks that are more representative of user behavior - e.g. focusing on under specified queries.↗28Uni 1New model from Luma that is a single, decoder only, autoregressive transformer that interleaves images and text on both input and output. As a result, it appears to have much stronger visual reasoning capabilities, and can also support much more complex input conditioning with long prompts & other image inputs. The controllability is particularly good.↗29After WIMPGood blog playing out how the fundamental assumptions on how web services and users interact is changing. Traditionally, the GUI/UI was the fixed exchange protocol between a user and a service. The future probably looks different - maybe some kind of UI/GUI scaffolding or principles, coupled with some kind of user-attached personal software libraries that outline preferences of that user, and dynamically compile into an application.↗30Challenges and Research Directions for Large Language Model Inference HardwareAmazing overview of how LLM inference merits a structural rethinking of chip & datacenter design. The insane growth of inference vs. training, coupled with novel architectures like MoE & long context, mean that the decode step of LLM inference is increasingly memory bound and latency constrained. They propose four major areas of research to address this - more of a focus on flash based memory vs. just DRAM/HBM, processing near memory (e.g. small compute units located closer to memory) for higher bandwidth, 3D memory stacking for higher bandwidth, and lower latency interconnect approaches.↗31UniFusion: Vision-Language Model as Unified Encoder in Image GenerationPaper from Adobe exploring using a VLM as the universal encoding layer for generative image models, replacing the more common separation of a vision + text encoder. Approach seems to result in models that generalize better and have better transfer in training, though it requires some nuance in how you apply VLM.↗32Interesting Directions in VisionGood overview of some recent trends in vision-language-action models for robotics. Main themes include: integrating tactile data, incorporating 3D reasoning and 3D priors, applying RL on top of base VLM/VLA, and unifying world models with VLAs↗33Agent Design Patterns OverviewIncredible overview of recent design patterns in agents. Resounding theme is adopting computing primitives as the basis set of tools (e.g. bash, file system, code) & offloading all context management to the computer↗34Recursive Language ModelsA proposed system architecture for models/agents, designed around recusively calling LLMs in a REPL environment where the context is represented as a variable in memory that is not shown in any way to the LLM unless it specifically asks for it using various tools (grep, peeking, etc). You essentially are asking the LLM to figure out how to probe & discern the context and identify how to manipulate it with a sequence of recursive sub-calls. In a way this is similar to how systems like Claude Code work, but they are coming more from the model design side vs. a task-specific solution. Amazing paper.↗35Towards a Science of Scaling Agent SystemsInteresting overview of how multi-agent vs. single-agent system design variations impact task quality. High variance of whether it improves or degrades in what situations↗36Automated Self-TestingGreat blog by Replit on agent design for automated testing of code gen agents. Particularly interesting was the design strategy of simply their computer use agent simply write playwright code rather than have specialized tools like select element, etc.↗37Sandbox RuntimeA lightweight agent sandboxing tool from Anthropic, based around filesystem and network restrictions using OS utilities vs. using full fledged container or microVM.↗38Everything is Context: Agentic File System Abstraction for Context EngineeringPaper exploring how the file system access can be an effective tool for context management. Similar to the Manus architecture.↗39Inside ThunderKittens' Python BindingsInteresting overview of part of the ThunderKittens, a framework for writing more performant GPU kernels out of Stanford↗40Radiance Fields and Future of Generative MediaGreat overview of the state of neural radiance fields and the role that 3D models will play in generative media.↗41Cheap RL Tasks Will Waste ComputeInteresting argument on why the world is shifting to extremely high-end, high cost, specific RL data away.↗42Principles of Diffusion ModelsHolistic overview of the principles of diffusion models.↗43Scaling Reasoning in Diffusion Large Language Models via Reinforcement LearningFirst example of applying RL to improve reasoning in diffusion models.↗44LLMs for Scheduling Policies in Distributed SystemsCool example of using an LLM + simulator to optimize a database scheduler. Generator + verifier pattern.↗45Barbarians at the GateInteresting overview of ideas for applying ML to systems research.↗46Supporting Our AI Overlords: Redesigning Data Systems to be Agent-FirstAgent-first data system design from Berkeley.↗47Improving Cursor Tab with Online RLOverview of how Cursor does online RL to improve their Cursor Tab model.↗48Denny Zhou – Reasoning SlidesGood slides on reasoning models and techniques.↗49SkyRL v0.1 — NovaSkyModular RL framework with separate trainer, environment, generator, and reward layers.↗50The Second Half of Machine LearningGood blog on how ML is moving from methods to environments and RL.↗51WeaverCombining multiple weakly supervised verifiers into a strong ensemble verifier.↗52BlockDiff Incremental VM SnapshotsCognition's OSS code sandbox designed for snapshotting.↗53TITANSAlternative autoregressive architecture with dynamic memory blocks for long context.↗54TAO: Test-Time Compute to Train Efficient LLMsCool example of using reasoning models to autonomously fine tune LLMs.↗55Scaling RL ComputeDiscussion of how to scale RL compute and its bottlenecks from General Reasoning.↗56Inductive Moment MatchingDiffusion-like method allowing discrete jumps in sampling and better use of pretrained networks.↗57Trellis3DUnified latent representation for generative 3D objects decoding to radiance fields, Gaussians, and meshes.↗58SIMATrains agents to act in diverse 3D game worlds from language inputs; notably it generalizes well across games where it has no game specific training.↗59Autoregressive Image Generation via Progressive UpsamplingTreats image generation as autoregressive refinement to higher resolutions, competitive with diffusion.↗60Flow MatchingGeneral method for training generative models via matching probability flows instead of noise corruption.↗61ChameleonMixed-modality model encoding text and images in a single token space trained end-to-end.↗62Training Verifiers for Math ProblemsEarly OpenAI paper on training verifiers for mathematical reasoning.↗63MuZeroModel-free RL algorithm mastering games without explicit rule knowledge.↗64Beam SearchUsing beam search as a test-time reasoning strategy.↗65Tree of ThoughtsReasoning strategy that generates candidate thoughts and explores them via tree search heuristics.↗66LLM ReasonersUnifies reasoning as reward technique, world model, and search algorithm; shows search and RAP outperform basic CoT.↗67Reasoning with Language Model is Planning with World ModelCombines a generator and a world-model LLM with MCTS for iterative reasoning.↗68Large Language MonkeysDemonstrates large gains from sampling many outputs and selecting via a verifier.↗69Beyond A*Trains transformers on A*-generated traces to internalize search-like problem solving.↗70Stream of SearchShows that post-training on search-style reasoning traces greatly improves CoT performance.↗71DualFormerTrains on full and partially masked reasoning traces to enable fast vs slow reasoning modes.↗72Training LLMs to Reason in a Continuous Latent SpacePerforms reasoning in latent space instead of over discrete token sequences.↗73Byte Latent TransformersTokenizer-free transformer operating on bytes with entropy-based dynamic patching.↗74The State of Generative Models - 2024 ReviewOverview of late-2024 trends in multimodality, reasoning, tokenization removal, and agents.↗75Model Context ProtocolAnthropic's framework for standardizing tools for LLMs.↗76FastMCPCool framework from Prefect that simplifies building production MCP servers↗77BrushWebGPU and Burn-based engine for training and rendering Gaussian splats.↗78UnboundedPrototype "infinite" game powered by distilled LLMs and diffusion models.↗79lolcatsConverts transformers into linear/state-space style models via attention replacement and LoRA.↗80SQLite in Durable ObjectsSynchronous embedded SQLite inside Cloudflare Durable Objects for session backends.↗81Differential TransformerInteresting idea of modulating attention to a relative score, instead of absolute, in theory reducing attentinon towards irrelevant context.↗82Networks of NetworksCool paper demonstrating how simple compound AI system designs (e.g. judge/verifier, best of K voting) can produce huge performance deltas.↗83MaestroNetflix's JSON-based workflow orchestrator supporting DAG and cyclic graphs.↗84Resource Management for Aurora ServerlessSome interesting discussion on resource and memory management in Aurora Serverless.↗85OpenHouseLinkedIn's open-source control plane/catalog for lakehouse architectures.↗86Exploiting Cloud Object Storage for High-Performance AnalyticsPaper exploring opitmal system design for querying cloud object stores efficiently.↗87The Architecture of Serverless Data SystemsAmazing six-part series on serverless data system design (Aurora, Dynamo, Neon, Kora, etc.).↗88Apple Intelligence OverviewOverview of Apple Intelligence system design & architecture.↗89Accelerating Code Migrations with AIGoogle deep dive on techniques for AI-assisted codebase migration.↗90Efficient finetuning of Llama 3 with FSDP QDoRAAnswer.ai blog an their "continued pre-training" method, QDoRA↗91Hybrid ML + Numerical Weather ModelCombines ML with traditional numerical weather prediction for long-range forecasts and uncertainty bounds.↗92Privacy in Public + Private RetrievalExploration of how to handle retrieval in AI systems assuming mix of private + public data to retrieve over.↗93Dynamic Partitioning for VisualizationTechniques for dynamic partitioning in data visualization.↗94Draco 2Renderer agnostic data visualization format designed to allow for flexible encoding of visualization rules.↗95DynaVisCool idea to dynamically synthesize data visualization editor widgets based on the data visualization task.↗96Formalizing Visualization Design Knowledge as Constraints: Actionable and Extensible Models in DracoFoundational paper on the Draco constraint-based visualization system.↗97SWE-AgentBenchmark for coding agents.↗98In Defense of Dual-Encoders for RerankingExplains why dual encoders underperform cross encoders and how to fix them.↗99Scaling MonosemanticitySeminal Anthropic blog about how to identify semantic features and their associated neurons in deep neural networks.↗100GPUs Go BrrrDiscusses GPU kernel optimization and hardware-aware AI system design.↗101GenieSuper interesting idea of inferring not just videos but action controllable worlds from input images. You train self-supervised on existing video data, and you learn both how to predict the next frame and the action that would have connected those two frames.↗102Foundation Models for Reasoning on ChartsCool applications of foundation models for reasoning about data visualization charts.↗103MeerkatExploration of a dataframe library that natively supports unstructured data types.↗104Reka Core / Edge Tech ReportTechnical report on Reka Core and Edge models.↗105GorillaX Exec EngineInteresting tool-use runtime that natively supports various ideas like undo.↗106RAFTFine tuning strategy to optimize for domain specific RAG workflows.↗107FASTERHigh-performance key-value store with an elegant log-structured design.↗108GarnetMicrosoft's high-performance KV store related to FASTER.↗109LIDASystem for AI-assisted data storytelling and visualization.↗110Mechanistic Design & Scaling of Hybrid ArchitecturesInteresting paper highlighting ways to mechanistically test models in small scale, specific tasks in ways that predict scaling properties, allowing for many architectural approaches to be tested rapidly↗111LumiereVideo generative model with improved temporal–spatial consistency.↗112Large Sequence Models for Software EngineeringCode models trained on the software engineering process (reviews, debugging, etc.).↗113Scaling Data-Constrained Language ModelsDiscusses strategies for scaling models when high-quality data is limited.↗