How to read (and reproduce) papers

Why read papers at all

Blog posts and videos are compressions of papers, and compressions lose exactly the parts you need when something breaks: the assumptions, the ablations, the footnote that says the trick only works above a certain scale. Reading the primary source is slower per paper but faster per unit of understanding, and it is the only way to follow a field whose half-life of "best practice" is measured in months. The goal is not to read everything. Andrew Ng's advice is to compile a short list of papers, skim all of them to about ten percent, and then decide which few deserve a deep read; two or three papers a week, steadily, beats a binge.

Where papers live

• arXiv is the preprint server where almost everything appears first. The relevant categories are cs.LG (machine learning), cs.CL (NLP and LLMs), cs.CV (vision), cs.AI and stat.ML. Browse the daily listings sparingly; it is a firehose.
• Hugging Face Papers curates a daily, upvoted shortlist with links to models and code. It is the best low-effort filter in 2026.
• alphaXiv adds discussion and AI-generated overviews on top of arXiv, useful for a first orientation before you commit to a full read.
• Semantic Scholar and Connected Papers answer "what came before and after this paper?", which matters because a 2020 method is often superseded by a 2024 follow-up you should read instead.
• OpenReview hosts the peer reviews for ICLR and NeurIPS; reading the reviewers' objections teaches you how experts critique.

The three-pass method

S. Keshav's classic note describes three passes, each with a decision at the end.

Pass one, five to ten minutes. Read the title, abstract and introduction; read only the section and subsection headings; read the conclusion; glance at the references and tick the ones you already know. Then answer the five Cs: Category (what kind of paper is it: a new method, a measurement, a theory, a system?), Context (which prior work does it build on?), Correctness (do the assumptions look reasonable?), Contributions (what does it claim?), Clarity (is it well written?). Decide: discard, keep for pass two, or file as background.

Pass two, about an hour. Read the whole paper but skip proofs and derivation details. Look hard at every figure and table: are the axes labelled, are there error bars, is the baseline tuned as carefully as the proposed method? Mark unread references that seem important. Decide: do you understand it well enough to summarize it to a friend with evidence? If not, and it matters, go to pass three.

Pass three, four or five hours. Virtually re-implement the paper: make the same assumptions and re-create the work in your head or in code, then compare with the authors' version. This exposes hidden assumptions, missing details and the paper's real strengths and weaknesses. You should finish able to reconstruct the structure from memory and to say which ideas you would reuse.

Andrew Ng's reading questions

In his Stanford CS230 lecture, Ng recommends reading a paper in several passes too, with a specific order: title, abstract and figures first (in many deep learning papers the figures contain the whole story); then introduction, conclusion, figures again, and a skim of the rest while skipping related work; then the full paper minus whatever does not make sense; and only then the math. At the end you should be able to answer four questions: What did the authors try to accomplish? What were the key elements of the approach? What can you use yourself? What other references do you want to follow? For the math, his advice is to re-derive the key results from scratch on paper; for the code, run the released implementation first, then re-implement it from a blank file.

Decoding a math-heavy paper

1. Build a notation table before anything else: every symbol, its type (scalar, vector in $\mathbb{R}^d$, matrix in $\mathbb{R}^{n\times d}$, distribution) and its meaning. Half of the apparent difficulty of papers is unstated notation.
2. Find the objective. Almost every ML paper minimizes a loss or maximizes a likelihood or an expected reward. Write it as one line, for example $\mathcal{L}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}}\left[\ell(f_\theta(x), y)\right]$, and ask what the paper changed: the data $\mathcal{D}$, the model $f_\theta$, the loss $\ell$, or the optimizer.
3. Check dimensions of the central equation; mismatched shapes reveal misread subscripts instantly.
4. Re-derive one step rather than all of them: the step where the paper's novelty enters. For DPO (8.3 Alignment: RLHF, PPO, DPO, GRPO and Constitutional AI) that is the substitution of the closed-form optimal policy into the Bradley–Terry loss; for DDPM (8.9 Generative models II: Diffusion and flow matching) it is the closed form of $q(\mathbf{x}_t\mid\mathbf{x}_0)$.
5. Read the ablations and the appendix. The appendix is where hyperparameters, failed variants and honest caveats live.

Intuition. A paper is an argument with an equation at its center and experiments as evidence. If you can state the equation, name the evidence and list the assumptions, you have read it; if you cannot, you have only looked at it.

Reproducing results

Reproduction is the pass-three of pass-threes and the fastest way to real expertise. Start small: the smallest model and dataset the paper reports, or a scaled-down proxy (character-level Shakespeare instead of OpenWebText, CIFAR-10 instead of ImageNet). Expect numbers to be slightly off; a reproduction within a few percent is a success, and a reproduction that fails teaches you which detail the paper omitted. Karpathy's nanoGPT and llm.c, Sebastian Raschka's LLMs from Scratch and the annotated implementations at labml.ai show the style: one readable file per idea, with the paper's equations in comments. The Harvard Annotated Transformer is the canonical example of a paper rewritten as runnable code. When a paper releases code, read it after your own attempt, not before; the diff between the two is where your misunderstandings hide.

Keep notes you can search

Use one template per paper: problem, method in one paragraph, the key equation in your own notation, results that matter (with the baseline), limitations, and "what I would try next". Writing the summary from memory is active recall (0.1 How to use this path (and how to actually learn)) and makes the knowledge yours. Store the notes where you can search them (Obsidian, Notion, a Git repository of Markdown files), and link papers to the lessons in this path where they belong. A paper club with two or three friends, one paper a week, is the single most effective accountability device for this habit.

Where this shows up later

• The ten papers in 11.2 Landmark papers: the reading list are your first three-pass reading list, and the timeline page lists seventy more.
• Project 15 in 11.3 Portfolio project ladder is a full reproduction; 7.8 Build GPT from scratch (nanoGPT walkthrough) and 8.13 Project: fine-tune a small LLM with LoRA and ship a chat app are guided reproductions you have already met.
• The frontier topics in 11.6 Frontier 2026: where the field is heading are only readable from primary sources, because the blog posts have not been written yet.