The Art of Reading Research Papers

Ali Madooei
School of Computing Science
Simon Fraser University
amadooei@cs.sfu.ca
http://www.cs.sfu.ca/∼amadooei

Abstract

This brief “how-to” guide will provide you with some instructions for reading
scientific research papers. Although there is no one way or right way of doing
this, you may find the guidelines helpful. In particular, if you are a beginner
graduate student in the field of computer science.

1 Introduction

I am going to write about the act of reading a research paper which every researcher has to do a lot.
It is a skill that is very different from reading a textbook or studying lecture notes, mainly for the
reasons that these papers are written in a highly technical and very condensed style because of page
limitations and that the intended audience is assumed to already know the area well.
Also, when reading a research paper, the goal is to understand the scientific contributions the authors
are making. This is not an easy task. It may require going over the paper several times, and probably
even looking up other papers for some of the details.
Reading research papers is the bread and butter of grad school; graduate student might even read
research papers for reasons other than their research, such as to review them for a seminar class or
for a literature survey of a new field. In these situations, you may not have the time to read the paper
in its entirety or several times to extract all the nuances.
For all these reasons, reading a research paper can be challenging; the good news is there are some
simple guidelines that once you learn and apply them, will make the process much more efficient.
There is no correct way of reading research papers. Those of you who have had significant experi-
ence reading such papers will have developed a style of your own. For those of you who have not, I
will discuss one approach to battle the impediments of this task. This is a strategy that requires three
stages: Engage, Siege and Conquer!
To develop an effective reading style for research papers, it can help to know two things: what you
should get out of the paper, and where that information is located in the paper. We will discuss this
next.

1.1 The structure of scientific research papers

A typical research paper contains the following sections:

• Title - every paper has a title and a list of authors with their affiliations.
• Abstract - A very brief overview of the paper which says what is in it and allows readers
to see if it is relevant to them.
• keywords - List of terms which describes papers’ ‘general subject’, ‘specific subject’, etc.

1
 • Introduction - This is where authors outline what the paper is about, what did they do
and why! It is usually structured as to provide a precise problem definition, overview of
state-of-the-art solutions (if any), goal of the paper, key ideas, main contribution and a short
outline.
• Materials and Methods - This section describes the proposed solution/method while de-
tails the contributions. It also explains how the study was conducted, what materials, data,
procedure, methodology, etc. was used in the study.
• Experiments and results - The section describes the experiments conducted and highlights
the findings and ties it in with how successful the proposed solution has been in achieving
what the study set out to do in the first place.
• Discussion - This is where the authors elaborate upon their findings, and explain e.g. What
the implications of their findings are, if it supports or contradicts established or provisional
methods, etc.
• Conclusion - Briefly reviews the problem and outlines the key ideas and main contributions
of the study. It might further emphasize the importance of the results in the field, and ties it
in with the previous research. It is also common to outline limitations of the approach and
plan future work.
• References - A bibliographic list of citations, documenting all of the sources that is used
for this research or that has acted as evidence of their work.

Probably the least effective way of reading a paper is reading from title to references. In fact, it can
be useful to read the paper “out of order” and skip certain sections. Let us now describe a strategy
to efficiently read research papers.

2 Step 1 Engage!

Begin by reading the introduction (not the abstract1 ), then glance through the pages just to look at
any illustrations, pictures and plots, and finally read the final summary and conclusions. Now, set
the paper aside and ask yourself a number of questions:

• What is the paper about?

• What problem does it claim to help solve?
• Why the problem is important? (motivation, challenges)
• What is the proposed solution? (hypothesis, key idea)
• Is the problem or the solution of interest to you?
• What does the paper accomplishes? (main contribution, important findings)
• If you were to tackle this problem, could you think of other (or better) solutions?

If you ask yourself these questions you are engaging with the paper in a way that you do not when
you are reading a textbook!
Notice this quick skim will take you just a few minutes and it is often the decisive factor on whether
you actually want to dig deeper into this paper or instead find a more interesting one to read.
Whether you decide to read or put this paper aside, it will be useful for future references if you
make a note and describe the paper using few sentences (preferably with a non-technical language)
based on your answers to above questions. In fact, try to produce an ‘unstructured’ (a.k.a. limited,
indicative) abstract. This is a very brief overview of the paper (one paragraph of 150 to 200 words)
that summarizes and communicates the research reported in the paper to other researchers2 .
1
The abstract allows you to find out whether the paper is relevant to your research. However, consulting the
abstract alone is often inadequate for scholarly purposes even for engaging with the research reported there.
2
That researcher is you, amonth later when you have forgotten everything about this paper!

2
3 Step 2 Siege!
If you decided to dig deeper, you need to read the paper with greater care but at this stage ignore
details such as theorem proofs. Your aim should be to try to understand the key ideas of the paper.
It is very common to get bogged down by technical or mathematical details and fail to see the big
picture.
Here is a strategy: read the paper front to back but leave out any equations or complicated descrip-
tions, so that you don’t slow down your progress through the paper. It can help if you print out
the paper and grab something to write and highlight with (put a question mark next to things you
don’t understand and move on, circle words you do not know, jot down further ideas, questions or
criticism).
Research papers are usually presented in the conferences before they find their way to a journal. If
you are reading a journal paper, it might be useful to find the conference version as it is usually a
shorter read. Also, it is worth checking the authors website looking for presentation poster or slides.
Reading is an active task; as you read the paper (or after you finished) ask yourself some more
questions:

• Does the paper propose a method in which case, is it sound? does it work?
– What are the main contributions? key ideas?
– How does their work fit in with other similar works?
– What improvement/extensions do they contribute?
– What are the main assumptions? Do they appear to be valid?
– What are the limitation of the approach?
– What are the novelties/strengths?
• How is the method evaluated?
– Are there any experimental/analytical errors?
– How the authors demonstrate/prove that their solutions work?
• What are the findings?
– Do their results make sense?
– Are the findings supported by persuasive evidence?
– Is there an alternative interpretation of the data that the authors did not address?
– How are the findings unique/new/unusual or supportive of other works in the field?
– Do the authors do what they said they are going to do?
• What is the most important figure?
• What are future direction of this research?
– What are some of the specific applications of the ideas presented here?
– Can the research results be applied to another context?
– What are some further experiments that would answer remaining questions?
• Is the approach something that you might use in your work?
– How do these results relate to the work you are interested in? To other work you have
read about?

Note that you can generally answer all of these questions without understanding a single equation!
“Writing is thinking” as David McCullough said “To write well is to think clearly. That’s why it’s
so hard”. In order to siege the paper and force it to surrender and reveal its secrets you shall write a
summary note at this stage as well. Organize your thoughts by writing a structured abstract for the
paper. A structured (a.k.a complete or informative) abstract contains a deeper and more extensive
overview of the paper that outlines the main points including for example assumptions made, argu-
ments presented, data analyzed, main results and conclusions drawn. Structured abstracts are still
short (typically less than 500 words) and usually formatted with any of the following subheadings:
Background, Objectives, Methods, Results, Conclusions. In your notes, also write down your ques-
tions and comments such as any limitations or extensions you see for the ideas in the paper, your

3
opinion of the paper (primarily, the quality of the ideas and its potential impact). It is also useful
to make note of most important figures/tables/etc. and interesting references to follow upon. Your
notes can act as a stand-alone entity instead of the paper for your future references.

4 Step 3 Conquer!
In order to understand a paper, you often only need to go through step 1 and 2; most papers surrender
after you Engage and Siege! Sometimes however, you have to get in, delve deep and Conquer!
Here is a strategy: read the paper much more carefully, trying to work through all the nitty-gritty de-
tails. This will involve looking up points that were not fully explained by consulting the references,
textbook, online resources (such as Wikipedia), etc.
It is very helpful if you take note as you read; work out the math, make sketches, draw figures,
visualize the procedures. Check the authors website, sometimes they put supplementary materials
which can include data and code. When possible run the code and redo the experiments. The best
is to implement the method yourself. I don’t know a better way to completely understand a method
other than implementing it yourself.
It also might be useful to follow on authors’ line of work in particular looking at the publications
that proceeded and succeeded this one. Note that online tools such as Google Scholar or Microsoft
Academic Search allow you to find out which later (more recent) papers have cited this one.
Above all, be critical. Read the paper as if you were asked to review it. As a reviewer you need
to evaluate the paper in term of its contribution, originality, strength and weaknesses. Put the paper
into context (how this article relates to other work in the field. How it ties in with key issues and
findings by others, including yourself) and consider its significance (impact to the field; importance
to your own work). Ask yourself:

• Is the research problem significant?

• Are the contributions significant?
• Are the claims valid?
• Is the actual execution of research correct?
• Are the correct conclusion drawn from the findings?
• What is a good argument against the case made by the authors?
• What is already known about this topic?
• What does this study add?

Be aware, reviewing is a challenging task: to form a truly educated opinion on a scientific subject,
you need to become familiar with current research in that field.
Remember: dont feel unworthy if you dont understand all of the details! The fact is, there will be
things you simply do not understand because either you do not have the adequate background or
they are just too complicated. Dont be afraid to ask questions of people who may know.
Understanding a research paper at depth might require reading it several times. Besides, looking at
references can become a recursive task as you will traverse from references of one paper to another
in order to make sense of the first one. You can end up spending months learning about a particular
research.
Reading a research paper can take a very long time when you start your research, but don’t worry,
it will go much faster as you gain experience. If you find yourself stuck, always ask yourself: Am
I spending too much time on this work? Have I gone back to read an article or learn a method that
would help me understand this work better?

5 Acknowledgement
I am grateful to all who have helped me to read research papers over the years. In particular to
Professor Sir Mike Brady and Professor Julia Schnabel (both from Oxford University) who last

4
summer put this wonderful reading group together as part of the Medical Imaging Summer School
2014 which I was attending. There is a link to that event in the references if you are interested, and
some more links to procrastinate on how to read research papers!

6 References
• Mike Brady, Julia Schnabel, “Medical Imaging Summer School: Reading Group Instructions”, Avail-
able at http://iplab.dmi.unict.it/miss14/ReadingGroup.html, 2014, Accessed
on Aug 2014.
• Philip W. L. Fong, “Reading a Computer Science Research Paper”, Available at http://pages.
cpsc.ucalgary.ca/˜pwlfong/Pub/inroads2009.pdf, 2009, Accessed on Aug 2014.
• Tia Newhall, “Some tips for reading research papers”, Available at http://www.cs.
swarthmore.edu/˜newhall/cs97/s00/ReadingAdvice.html, 2000, Accessed on
Aug 2014.
• Bill Griswold, Gail Murphy Christina Conati, Charles Rich, “How to Read a Research Paper”, Avail-
able at http://web.cs.wpi.edu/˜rich/courses/howto.html, 2008, Accessed on Aug
2014.
• Robert Siegel, “READING SCIENTIFIC PAPERS”, Available at http://web.stanford.
edu/˜siegelr/readingsci.htm, Accessed on Aug 2014.
• Mary Purugganan, Jan Hewitt, “How to Read a Scientific Article”, Available at http:
//www.owlnet.rice.edu/˜cainproj/courses/HowToReadSciArticle.pdf
2004, Accessed on Aug 2014.
• Michael Mitzenmacher, “How to read a research paper”, Available at http://www.eecs.
harvard.edu/˜michaelm/postscripts/ReadPaper.pdf, Accessed on Aug 2014.
• S. Keshav, “How to Read a Paper”, Available at http://blizzard.cs.uwaterloo.ca/
keshav/home/Papers/data/07/paper-reading.pdf, 2013, Accessed on Aug 2014.

A great guide on reviewing research papers:

• Alan Jay Smith, “The Task of the Referee”, Available at http://www2.cs.sfu.ca/˜mori/

courses/cmpt888/summer10/papers/smith_taskofreferee.pdf, 1990, Accessed
on Aug 2014.

A must-see video on writing research papers!

• Simon Peyton Jones, “Seven Simple Suggestions: Don’t Wait - Write”, Available at https://
www.youtube.com/watch?v=g3dkRsTqdDA, 2013, Accessed on Aug 2014.