Can Diffusion Replace Autoregressive LLMs? The Complete LLaDA 2.X Guide
From DDPM to LLaDA 2.1 -- everything about diffusion-based LLMs. Masked Diffusion, Token Editing, and MoE scaling dissected across 4 parts.
Can Diffusion Replace the LLM? A Complete Anatomy of LLaDA 2.X
ChatGPT, Claude, Gemini — every large language model (LLM) we use today is built on a single principle. Autoregressive (AR) generation: left to right, one token at a time, predicting the next word.
This approach works remarkably well. But it has structural limitations.
- Tokens must be produced one at a time in sequence, making parallel generation impossible
- Even if the model knows "A is B," it cannot infer "B is A" — the Reversal Curse
Related Posts
MiniMax M2.5: Opus-Level Performance at $1 per Hour
MiniMax M2.5 achieves SWE-bench 80.2% using only 10B active parameters from a 230B MoE architecture. 1/20th the cost of Claude Opus with comparable coding performance. Forge RL framework, benchmark analysis, pricing comparison.
Backpropagation From Scratch: Chain Rule, Computation Graphs, and Topological Sort
How microgpt.py's 15-line backward() works. From high school calculus to chain rule, computation graphs, topological sort, and backpropagation.
Karpathy's microgpt.py Dissected: Understanding GPT's Essence in 150 Lines
A line-by-line dissection of microgpt.py -- a pure Python GPT implementation with zero dependencies. Training, inference, and autograd in 150 lines.