PHYSICS AND ENGINEERING RESEARCH › READING A PAPER
How to Read a Paper You Are Not Ready For
Nobody reads a research paper the way they read a chapter. A paper is not written to be understood on the first pass, and being lost on page one is not a sign you should not be reading it. It is a sign you are reading it in the wrong order.
R.A Say what question each section of a paper answers, and read a paper in an order other than front to back.
R.B Extract the claim, the evidence and the limitation from a paper whose methods you do not fully follow.
R.C Judge a source before you rely on it, and cite it properly.
1. A paper answers five questions, in a fixed order
Every research paper you meet this year will have roughly the same skeleton, and each section exists to answer one question. Knowing which question is which is most of the skill.
The abstract answers: what did we do and what did we find? It is the whole paper in a paragraph, and it is written last.
The introduction answers: why did anyone need this done? It ends with the specific gap the paper claims to fill, and that last paragraph is usually the most useful sentence in the paper.
The methods answer: what exactly did you do, precisely enough that I could repeat it? This is the section you will understand least, and the one you can most safely skim on a first pass.
The results answer: what happened? Numbers and figures, with as little interpretation as the authors can manage.
The discussion answers: what does it mean, and what are we not entitled to conclude? The limitations paragraph lives here, and it is often the most honest writing in the document.
Which section answers this?
A question a reader might have. Pick the section of the paper that answers it. The feedback explains what that section is for, so a wrong guess still teaches you the structure.
2. Three passes, and what you are allowed to skip
First pass, five minutes. Title, abstract, section headings, every figure and its caption, conclusions. At the end you should be able to say what the paper claims and roughly how they argued it. If you cannot, stop and read the abstract again rather than pressing on.
Second pass, half an hour. Read the whole thing but ignore anything requiring mathematics you do not have. Mark what you did not follow instead of stalling on it. Note references you would need to read to understand the parts you skipped.
Third pass, only if you need it. Reconstruct the work. Question every assumption, and try to see how you would have done it. This is the pass you will use for the paper you present, and for almost nothing else.
3. Figures carry the argument
In most experimental papers the figures are the paper and the text is commentary. Learn to read one properly and you can extract the result without the prose.
Read a figure in this order: what is on each axis, including units, then what one point or one line represents, then the error bars, and only then the shape. The shape is the part your eye wants first and it is the part that misleads you first.
Watch the axes. An axis that does not start at zero can make a two percent difference look like a revolution, and it is not necessarily dishonest to do that, but you have to notice. A log axis turns exponential growth into a straight line, which is why it is used, and it also flattens differences that matter.
4. Judging a source before you lean on it
Not everything that looks like a paper is one. Before a source goes into your presentation, ask four things.
Was it peer reviewed? A preprint has not been. That does not make it wrong, and many important results appear as preprints first, but it means no independent expert has checked it.
Who paid for it? Funding does not invalidate a result. Undisclosed funding by a party with an interest in the result is a reason to read the methods much more carefully.
Has anyone else found the same thing? One paper is a claim. A claim replicated by people who did not want it to be true is knowledge.
How old is it, and does that matter here? A 1960s measurement of a fundamental constant may be superseded. A 1960s argument about a concept may not be.
Would you rely on this source?
A source, described honestly. Decide whether you would cite it as evidence in your colloquium, cite it with a caveat, or leave it out. There is often more than one defensible answer, so read the feedback even when you are right.
5. Notes that survive the week
Take notes in a form you can use later, which means not highlighting. Highlighting feels like learning and produces nothing you can search.
For every paper, write four things in your own words: the claim in one sentence, the evidence in one or two, the limitation the authors admit to, and the question it left you with. Four lines. If you cannot write the claim in one sentence you have not finished the first pass.
Record the citation the moment you open the paper, not at the end of the project. Authors, year, title, journal, volume, pages, and the DOI. The DOI is the only one of those that will still find the paper in ten years.
Check yourself
1. You have twenty minutes and a paper you have never seen. Describe exactly what you read, in order, and what you should be able to say at the end.
Title, abstract, then every figure with its caption, then the final paragraph of the introduction, then the discussion and conclusions. You should be able to state the claim in one sentence, name the kind of evidence behind it, and name one limitation. You should not expect to be able to reproduce the method, and you should have a written list of what you could not follow.
2. A figure shows a bar chart where one bar is visibly twice the height of the other, and the vertical axis runs from 48 to 52. What has happened, and is it dishonest?
The suppressed zero has magnified a difference of a few percent into a doubling. Not necessarily dishonest: when the interesting variation is small, expanding the axis is the only way to see it, and the axis is labeled. It becomes misleading when the caption or the text describes the result in language that only the visual impression supports. The check is to read the numbers off the axis and restate the difference yourself.
3. You find a preprint with a result that is perfect for your colloquium, and no other paper reports anything like it. What do you do?
Use it, and say what it is. Present it as a single unreplicated preprint rather than as established, and say so out loud in the talk. Then look for who has cited it since, and look specifically for anyone who tried and failed to reproduce it. The problem is never citing a preprint. The problem is citing one as though it were settled.
4. Your notes on a paper are three highlighted pages. A week later you need the paper’s main limitation. Why is this hard, and what should you have written?
Highlighting records where your attention was, not what you concluded, so recovering the limitation means rereading. Four lines in your own words would have done it: claim, evidence, limitation, question. Writing in your own words also exposes what you did not understand, which highlighting conceals from you.
Working With Data in Python. Reading somebody else’s results is one half of the job. The other half is producing your own, and that starts with getting a real data file onto a screen and asking whether the number that comes out of it is believable.