Advice for scientific writing
Denis G. Pelli
Psychology and Neural Science, New York University
July 29, 2015
Collaborating on manuscripts, I often find myself repeating the same
recommendations, so perhaps it will be useful to write them down in one place,
where they might be consulted by those who would like some guidance on style.
Some of these comments are meant for students writing their first scientific
essay, and may seem obvious to more experienced writers, but they’re all applicable
to all my scientific writings. These are my opinions, but I learned a lot about
how to make a good figure from John Robson, Beau Watson, and Tony Movshon.
0. Having an idea. Mentoring high-school science projects, and teaching Lab in Perception and Experiments in Beauty to NYU undergrads, I've helped hundreds of students to formulate an experimental question in perception, reduce it to an experiment, and write up a report. The beginning can be vague or half baked: wanting to measure an effect, or curiosity about how something works. It's good to actually try something experimental, and it may help to ask Google scholar what's been done before. Hopefully in a few weeks you'll have a concrete question (e.g. What is the effect of X on Y?), a method, and some idea for how you can draw a strong conclusion from imagined results. Science is reductionist, because, to convince, you have to focus your efforts, testing the simplest possible story. Hopefully, the reduced question still interests you and seems likely to interest others. In the end, to do science, you'll have to convince someone else of your conclusion, which requires that your title be interesting enough for her to continue reading.
0.5 Ask each participant to sign your university-approved consent form. Don't forget. You need this for any data that will be published. You don't need it for unpublished pilot data. Here are the NYU criteria for human subjects research, which needs approval by the University Committee on Activities Involving Human Subjects (UCAIHS).
1. Convince. You must convince the reader of an interesting new conclusion:
Interesting, new, and true. If you fail to convince, then
you have no impact and no scientific contribution. Convincing your peers is an essential part of the modern definition of science. It's social, not private. By this standard, Leonardo was an artist, because he showed his paintings, but not a scientist, because his “scientific” notebooks were published only after his death. If you merely write in a private notebook, or publish in a way that convinces no one, your efforts may be admirable in other ways, but it's not science. What do I believe after reading your paper that I didn't know before?
2. Keep going. Don’t take your audience
for granted. Reading is hard and readers are impatient. They’d rather be doing
their own research. Each sentence must convince the reader to persevere.
One misstep and the reader will put down your paper, probably never to return.
3. Communicate. Ask a friend to read a draft (the whole thing or just the abstract) and then tell you the gist We often do this in lab meetings, asking the author to read her abstract aloud, and then calling on someone randomly to say what it's about. It's a dramatic test of how well the abstract is working.
We often talk of papers being great, as though it were an intrinsic quality
independent of the readers. In fact, papers have value only to the extent that
they succeed in communicating ideas to their readers, the particular audience
that you are trying to reach. Understanding anything new is deucedly difficult.
Thus, what strikes the author’s
ear as perfect may in fact be inferior to a plainer longer exposition that
is easier to read. Polishing should heed reader complaints, especially
what they don’t get. Interrupting the reading with questions, asking
the reader to paraphrase, may reveal what the reader missed because the text fails
4. Be friendly. Most scientists would like to publish in Science and Nature,
but the competition is so fierce that only what appear to be earth-shattering
discoveries are accepted. This has the unfortunate side effect of encouraging
authors to write everything as a breathless revelation. Hard things
are presented as though obvious, and, alas, reviewers are too often unduly
impressed by what they do not understand. Personally, I know that it’s
easy to lose me, so I’m grateful and impressed when the author helps
me to understand. Sometimes it’s hard to be both clear and correct at
once, and a comment may help. To ease a hard passage, imagine the reader as
a cherished student, and whisper a stepping stone or gist.
5. Be complete. To be scientific, your paper must specify methods in sufficient detail for someone else to replicate your result. Beyond that, it is friendly is to provide complete results and methods. Nominally we provide just enough to prove our point and allow replication. But it's better science when we provide, within reason, the whole data set and relevant, if not strictly necessary, details. This allows the reader to use our data to address other questions, beyond those posed in our paper. Her question may be sympathetic, critical, or unrelated to our own. This exposure to expert readers helps make science much more reliable than any individual author can be.
6. Take credit. Specify exactly what is the contribution of this paper. The contribution should be specified in the abstract and the conclusion. This is both an obligation to acknowledge and an opportunity to brag. Spell out what this paper should be credited for. Don't be shy. You want the reader to understand the gist, getting the big picture in all its glory. Make it as short, clear, and exciting as you can. Don't hold back, but you must also make clear what is new, the fraction contributed by this paper, by acknowledging what came before. Your conclusion is rarely entirely your own. It usually builds on things that came before. You have an obligation to spell that out so the reader can distinguish what's new from what should be credited to other papers. The reader needs both the gist and the acknowledgment.
7. Draw the figures and put them where they belong. Most of us don’t
want to see the manuscript until you’ve included the figures (graphs,
etc.), but they needn’t be final data. Crude cartoons are fine. We need
to see a graphical expression of what the figure is meant to communicate. (Yes,
please, sketch something now.) Include the figures in the text where they’re
meant to be, not at the end. Some journals still have archaic rules demanding
that the figures be at the end of the submitted manuscript. This is for the convenience of the production
staff at the expense of the reviewers. However, in my experience, the journals
only enforce these rules at the end of the review process, so you can initially
provide an easy-to-read layout for review and, later, once the paper is accepted,
provide the specified layout for production. Number the figures. Every figure should
have a caption explaining what's in it. (Incidentally, in Microsoft Word, it is usually less troublesome to put the figures "inline" like text, rather than floating in a separate layer over the page. When you want to have the figure caption to the side of the figure, create a Table with one cell for the figure and one cell for the caption.)
8. Look at your figures. Reading graphs is a learned skill. As scientists
become more fluent at reading graphs, they look longer. Try looking at your figure for five minutes. What does it
seem to be saying as an image? Adjust the figure and the text to tell the same story. They are much more convincing when they are in harmony. You
can change anything except the data. These cosmetic changes, over many iterations,
will make your figure much more effective. Multiple graphs should be consistent,
providing a coherent account.
9. Show off your data, not chart junk. The purpose of graphs is not to create a box into which you cram information. The purpose is to communicate with the reader. The essential part of the message is the data. Not the rest of it: boxes, labels, ticks, legends, etc. The rest is necessary, but, compared to the data, it's junk. Don't shrink the data to make room for the junk. The junk should never compete with the data, either for space or for ink. It's like the bride at a wedding. No one should dress so as to upstage her.
10. What is the paper really about? I have several times been stunned, in writing our seemingly superfluous cover letter explaining to the editor
what's so great about the enclosed manuscript, to see ideas emerge that were
not explicit in the paper itself. Borrowing from the letter, the paper was
much improved. It's as though the story is revealed, not invented, and will
tell itself, if we only let it. So I recommend, when you think the paper's done, that you ask yourself what it's really about. The answer may surprise you.
11. Forget your hypothesis. The paper is not about whether you guessed right. I discover that many students writing their first paper attach too much importance to finding that their hypothesis is right. Often they celebrate finding that their data are compatible with their hypothesis, as though this were a competition and we got points for guessing right. Firstly, there is typically no public record of what the authors originally hypothesized, and authors often change their hypotheses to suit the scientific story that emerged from their data. (Yes, that's ok, and encouraged.) So it would be difficult or impossible to award points for guessing right. More important, confirming the hypothesis is not cause for celebration. If you're always right, your experiments aren't aggressive enough. Experiments are most informative when the outcome cannot be predicted. When you present a hypothesis that is consistent with past literature, and, after collecting your new data, that hypothesis is still viable, your paper may be in trouble. One can wonder whether you learned anything at all. To make progress, you must disprove something. Hopefully you can think of something, previously viable, that your new data disprove. The value of your experimental data is independent of your original hypothesis. The value of the hypothesis was in provoking the experiment. You can think of this as drawing a line in the sand. Imagine the space of all possible theories as a vast sandy beach and that your experiment tests a line in that space, showing that the true state of the world is to one side or the other of the line. It doesn't really matter which side of the line you guessed would be true. The value of your hypothesis (right or wrong) was entirely in provoking your test of that line.
12. Number the pages and the lines. Anything
you send for comment should have numbered pages. It is annoying, when writing comments or a review, to lack page numbers. It’s nice to indicate
the range: “Page 1 of 17”. Remember that many printers
don’t print the top or bottom 0.5 inch. You can make it even easier for your commentator by providing line numbers, from beginning to end. In
Microsoft Word, select
“File:Page setup:Microsoft Word:Margins:Layout:Line
13. Name the document file. The file should have a name based on the
title or first author’s name and should end in a number representing
the draft number, e.g. channels3.doc or martelli7.doc. (Some of us have many
manuscripts to keep track of.) Microsoft Word files have a much better chance
of surviving transmission through email if their filename ends in “.doc” or
“.docx”. (Since you’re going
to be emailing the file, it’s best if its name has no spaces or underscores.
Run the words together, or use a dash to separate them, e.g. spatialfrequency.doc,
spatialFrequency.doc, or spatial-frequency.doc.) It’s a good idea to
mention the filename or at least the draft number on the first page, so that one can easily
tell which draft a paper copy represents. Remember to update this draft number every time you create a new draft. Anyone who edits the file
should increment the draft number before sending it to anyone else.
14. Read it aloud. Alex Holcombe mentions “the well-known finding that reading sentences aloud makes it easier to improve their writing. Use of our vocal apparatus manifests the natural articulated rhythms of text, which we might not register when reading silently.” In the brain, there are two streams of auditory processing of speech, one for comprehension and one for articulation (Hickok and Poeppel 2007). Reading aloud uses both.
15. Help the reader to cite you. There are many kinds of document today, e.g. blogs, and, if you want
to be cited, it’s best to show the reader how to cite your
work. Manuscripts should have a title, authors, and date
even when they are tentative and subject to revision. This is part of laying
claim to an idea. Put enough on your front page that someone
who receives it could cite it. The key things are title, authors, and date.
Until publication, the title and authors are subject to revision, but without
them, and the revision date, the document is almost impossible to specify.
16. Nudge collaborators. Collaboration is wonderful. The key ingredient is that
you both must need each other. That's what will carry you through the hard
patches. Alas, many manuscripts die sitting on the desk of someone
to get back to it soon. How do you get it moving again? This is often described
in moral terms specific to the personal association, but, after many years of
sending and receiving reminders, I've come to think that it's a professional skill.
Some people are good at it and they collaborate widely to produce many papers. Others aren't immoral; they just aren't good at it. Watching,
from both sides, what works and what doesn't, I note that there is a trick
to it. Start very very mildly, lightly reminding. And stay there. Don't escalate.
This is counter-intuitive because, as a sender, one is embarrased by the implicit
criticism of the reminder, and one feels a need to justify the action by moralizing
and describing dire consequences. But all that negativity discourages the
probably needs only the reminder and perhaps some encouragement. And, of course,
do it. Always very
lightly, but frequently enough to keep the paper in your recipient's mind.
Mastering this unsung skill and collaborating with good nudgers — nudging and being nudged — may greatly increase the number of papers that you publish.
17. Understand the reviews of your submitted manuscript. Receiving journal reviews that don't recommend acceptance is distressing. The comments may seem stupid and mean. I suppose some are, but mostly they are the best attempts of people just like us, unpaid volunteers, struggling to understand the paper and give sound advice to the journal editor and authors. Very slowly, over the years, I've been learning how to read reviews. It's worth spending enough time with the review to get to the point where you can put yourself in the reviewer's shoes and imagine writing the review that lies before you. When this succeeds, I suddenly realize that, oh, yes, if I were the reviewer and thought that way, then I would expect something else, not what we found, and be disappointed or even dismayed by what we said. At that point, having conjured up a viable model of the reviewer, I can consider how we might say things differently to include the reviewer and keep her on the path of our story. If this reviewer got it wrong, it's likely that other readers might too. Fixing it for the reviewer might grealy increase the number of readers who accept our conclusion. Sometimes it's just a matter of adding a few words, acknowledging a consideration or contrary assumption, or cautioning against a tangential attractive nuisance. Negative reviews are painfully hard to read, because we identify so much with our work. But the editor did pick the reviewers to be knowledgeable experts, so they are a sample from our target audience, and are thus an invaluable guide for how to increase the success of our paper in communicating. Even if it's wrong, the negative review is evidence of faulty communication that wants repair. If I can put myself in his or her shoes, I have a chance to whisper words into my paper that might bring him or her on board. Often the reviewer correctly senses a problem but suggests a remedy that seems worse that the original. In those cases, I try to understand the problem to find a better remedy that makes the paper better (not worse) while fixing the problem. The aim is for every change to make the paper better, while satisfying the reviewers.
When you write your letter responding to each point in the review, if you manage to satisfy a particular reviewer request or comment, it is helpful to begin your response to that request with "Done." If it's a misunderstanding, one might begin with "Clarified." A reviewer reading your response may be heartened to see a long list of Done and Clarified as an encouraging sign that the reviewing process improved the paper.
18. Tips for Microsoft Word. I use Word partly because it has features that I need (superscripts, footnotes, formatting, compare drafts), but mostly because it's popular and I write most of my papers in collaboration. It's clumsy, but it can do nearly everything needed. Here are a few tips.
18A. Don't use Word's comments. I often need to print the ms to take it with me, eg to dinner, and Word's printing of comments is miserable. it makes the body of the ms unreadably small. Just insert the comment in the text, and make it stand out by using ALL CAPS, highlighting, or bold italic. If you highlight, be aware that yellow highlight disappears when you print in black and white; use green highlighting instead.
18B. Use Word's linebreak options to format headings. It is ugly when a section heading appears at the bottom of the page. Word offers a convenient way to prevent that. Select the heading (and the blank line following it, if there is one) and use the menu item
Format:Paragraph:Line and page breaks:Keep together.
18C. Use in-line figures. Word offers two ways of placing figures in text. Inline works fairly well. The alternative allows you to drag the figure where you want it, and Word wraps around it. However, that kind of placement is fragile. In both ways you can fiddle to get the page to look right, but when you do it inline, it's robust, and stays ok. When you place with wraparound, the page is often screwed up after you edit earlier parts of the manuscript. I've wasted a lot of time with that fragile feature. It saves a lot of time to just use inline. Select the figure. Then, in the formatting bar, hit "Format Picture". Then click "Wrap Text: Inline with text."
18D. The built-in Help is often less helpful than posing your question directly to Google, prefaced by "Microsoft Word".
18E. NYU users can get Microsoft Office, including Word, for free. See Links below.
Title. One usually thinks of the title as a statement of scope or a memorable gist, inviting the reader and reviewer. However, note that when choosing what paper to cite, wrters will often choose a title that matches the point they are trying to justify, so that a concrete assertion (a sentence) may garner more citations than a generic topic (a subject). Of all the phrases in your ms, it is the title that has the greatest effect on editors, reviewers, and readers, so it's worth getting your friends to judge it, especially when you're trying to choose among several candidates. I've always asked, “Which do you like best?” but I've just discovered (in 2011) that this is the wrong question. Today, my friend liked the short sober title better, but, when asked which title was better at convincing her to read the abstract, she chose the long cutesy title, “by a large margin.” For getting published and read, convincing the reader to proceed is what matters, so, henceforth I'll ask about that instead of liking.
Authors. There are various widely used but inconsistent principles
for ordering the
authors’ names in the byline. I recommend strict descending
order of contribution, but you should
consider the expectations of your audience. Neuroscientists often put the student
of how you order the names, all the societies encourage you to specify, in
what each author contributed. I now do that in all my papers. My favorite article on this topic is Riesenberg,
D., & Lundberg, G. D. (1990) The order of authorship: who's on first? JAMA,
264(14), 1857. Several important societies comment on current practice: APA (8.12
Publication Credit), Society
for Neuroscience (1.3.3), PNAS, and ICMJE (Uniform
Requirements for Manuscripts Submitted to Biomedical Journals).
Abstract. Convince the broadest possible audience that this is interesting
and important. In a few sentences, tell us what you did and what you conclude.
Bear in mind that most people considering whether to cite this paper will assume
that all the conclusions are present in the abstract. It is my impression that most citations, today, are based on reading just the abstract. The main measure used today for scientific impact is still the citation count. Thus, to some extent, like it or not, for assessment purposes, your scientific contribution is your abstract not the paper. So consider how well your abstract holds up on its own.
As noted above, specify exactly what is the contribution of this paper. What will the reader believe after reading your paper that she didn't know before?
Citation. Getting cited is the name of the game in academic science. Make it easy for people to cite your work correctly by providing a formatted citation that the reader can copy to the reference list of her own paper, when she cites you. The citation will generate more traffic back to your article if you include a URL linking directly to your PDF. Typically the citation is placed below the abstract. If you're publishing in a journal, this will be done for you. But when you self-publish, e.g. putting your thesis on the web, you need to do it yourself.
Introduction. The main purpose of the intro is to motivate the work (i.e. convince the reader that this question is sufficiently interesting to be worth reading about), but this is also where you credit what’s already been done by others, especially by potential reviewers. The intro typically takes the form of a historical review, but that’s more a pretext than a purpose. The purpose is to motivate and give credit. If you are not yet well-known in this field it may be important to the reviewers that you show awareness of the key papers in the field. You can get this list by scanning the introductions of other papers. You needn't praise. It's enough to cite.
Methods. It’s important that this be correct, complete,
and understandable. It should enable the reader to replicate your experiments.
The writing can be ugly and repetitive, but it must be complete.
How will you feel if someone fails to replicate your result uh oh! because
you omitted an important detail? It is usually skipped (or lightly skimmed)
in a first reading, and consulted later, to look up details. The old tradition is to place methods in the middle of the paper. Experienced readers skim or skip methods on a first reading. The high-impact journals tend to relegate methods to the end. I'm coming to think that all journals should do that, so we wouldn't have to skip. Methods belong at the end. In psychophysics, we talk a lot about the "trial". A trial consists of the procedure and stimuli to collect one observer response. When describing a psychophysical trial it is usually far preferable to use the singular: Describe one observer seeing one target in one condition. Other trials may involve other observers, targets, and conditions but it is important that the reader correctly understand what happens on one trial, before you explain the variations. A big advantage of vision science over other fields is that we can often include our stimuli in the paper. I am forever surprised by how few papers do this. It is a great help to the reader to actually see your stimuli instead of merely trying to imagine them.
Results. Data. Graphs. The results text should
have a very plain style. “Just the facts, ma’m.” Only minimal interpretation
and comparison to other work. But do mention replication and inconsistencies
(real or merely apparent) with past work. Sometimes the empirical result
is more or less the conclusion of your paper. Sometimes that conclusion needs
a reasoned argument, which may appear here or in Discussion.
Discussion. Try to give the reader the big picture. Take a step back. Try to forget your stake in this and guide the reader through your garden, noting the various considerations, positive and negative, that seem relevant. Connect this work to that of others. Even distant connections help, as readers come from various places and it always helps to understand the connection, however distant, of what’s new to what’s familiar. However, the meandering connections, desirable as they are, are no substitute for a tight argument that forces the reasonable reader to accept your conclusions. Ultimately that’s the core of your contribution.
Conclusion. Most papers published in psychology
do not have a final section labeled “Conclusions”. My own view
is that it is rarely reasonable to publish a scientific paper without a conclusion,
and that it is helpful to draw attention to its presence by setting it off
in its own section. The conclusion should be short and as strong as you can
make it. I consider the conclusion to be the reader’s reward. This is where you deliver on your initial promise: What do I believe after reading your paper that I didn't know before reading it?
Acknowledgments. There’s a lot of freedom here, but try to be
concrete (what exactly was the contribution?) and flattering. People should
be glad to be thanked. If you can’t word it to achieve that effect then
don’t bring it
up. Specify what each author contributed to the paper (see Authors, above). It is customary to cite here any earlier presentations of these results at a conference or in a thesis.
References. It is almost impossible to type references by hand without introducing errors. Use Google Scholar. When it finds an article it gives you a "Cite" button. Select APS format. Copy that into your reference list.
Reference formatting. Reference lists are hard to read, and you can help the reader by using a good format. I suggest single spacing with hanging indent and a gap after each paragraph. In Microsoft Word, use Format:Paragraph:Indents and spacing, to set Indentation: hanging by 0.5"; Spacing: after 12 pt; Spacing single. This creates a gap between references without having to insert a blank line. That is handy because it allows you to use the Microsoft Word Table:Sort command to alphabetize all your references.
Limitations of this study & Future research. No! High school and college students are often told that they should include sections on limitations and future research, but published papers rarely do so, for good reason. Scientific papers rarely provide “suggestions
for future research,” because authors usually do not want
to share their best unexploited ideas, and it’s disingenuous to recommend
less-than-best ideas. High school students and undergraduates, writing their first paper, often use these sections to trash their own work, explaining how much better the paper might have been with more time and 20-20 hindsight. This “might have” stuff has no place in your paper. You should be reporting only what you actually did, and drawing conclusions from that. Your readers do not want to hear about what you might have done. It's boundless and boring. It is of course important for the reader to know the limitations of your conclusion, but most limitations are too obvious to mention. Everyone knows that you could have tested more observers and thus reduced the standard errors. Practically every study ever done could have been done bigger and better. Don't try the reader's patience by telling her what she already knows. It is important to specify the scope of the paper's conclusions, but this is usually done in a positive way in the abstract and conclusion. When new authors write these sections, the result is often painful to read, a long apology for what might have been. They damage the paper by killing the mood. Don't do it. Of course, if your conclusion is going to affect public health, you want to specify its limitations, but make that a positive part of spelling out your conclusion, not a separate apology.
Images (including stimuli) are important in articles on visual perception and can aid scientific understanding of the work. While respecting copyright law, there are five ways of publishing images, or access to them:
1. Under US law, the authors own the copyright for any images they create, and can publish them freely.
Images created by others can be published in the following ways (2-4):
2. Planning in advance, one can use images for which permission is already available, as Public domain or Creative-Commons licensed. Google Advanced Image Search offers the “Usage Rights” field to limit by license type. Flickr and Wikipedia have similar options. Here's a longer list of sources, under "Using images".
3. Writing for permission. Sometimes it is practical to obtain permission from the copyright holder in advance of publication, especially for scientific research. For example, the two Where's Waldo images in this article: http://www.journalofvision.org/content/14/6/10.full.pdf
4. URL. In lieu of the image, just provide a URL link to the original image at its original location on the web. It can be linked, so clicking takes the reader to the image. No permission is needed to publish the URL and link. Some of these links may break over time, but that is usually much better than not having access to the images at all.
5. Generally thumbnails (reproductions with a limited number of pixels) are allowed under the doctrine of "fair use" in circumstances such as those here, namely a scientific article in a free academic journal. Under the fair-use doctrine, courts have decided that it is ok to provide a thumbnail image (e.g. Google uses 150x150) to reference the full work. You can link your thumbnail to the original image at its original URL, so clicking will take the reader there. Your must add text crediting the source (see Ball State University guide for examples).
You can read more about the fair-use doctrine of copyright law below.
Figures in place. For production, most journals need to have the figures provided as separate files. However, it is much easier to review a paper with the figures in place, near where they are discussed, as in most publications. Alas, many journals tell you to submit your manuscript for review with the figures at the end. This is bad for the reviewers and you, because it makes your paper harder to read. You can often ignore this premature rule. I recommend submitting papers for review with the figures where the reviewer can easily see them. Editors understand this, and usually postpone enforcement of placement rules until the paper is accepted, at which time practically all journals need to have the figures in separate files. Unless, of course, they want you to produce the final version yourself, using your own page layout software.
Name your conditions. If you have multiple conditions, don't call them 1, 2, 3. Give them sensible names. This makes your graphs and tables much easier to understand.
Vector graphics vs. pixel-based images. An image file can describe an image in two ways, vectors or pixels. A vector-based image file is a list of commands that describe how to draw lines and curves. A pixel-based file is basically a table specifying the color of every point in a regular grid. Some images are suited to representation as vectors, and this is advantageous because that file is usually much more compact and its line edges are always sharp no matter how much you zoom in. Vector files are usually Postscript, with file type PDF, EPS, or AI. (Of these, only PDF can include the font description to allow printing of the file independent of the computer and font files used to create it.) Some graphing programs allow you to save your graph as a postscript PDF file; others allow you to "print" to PDF, creating a postscript file. Many programs will read postscript files and convert them to pixels; practically the only program that will retain the Postscript format is Adobe Illustrator. There are many popular file formats for pixels, including PNG, GIF, JPEG, and PDF. There are many good programs for editing pixel-based files, e.g. Graphic Converter and Adobe Photoshop. For production, all journals
now strongly recommend vector graphic files (i.e. Postscript) instead of pixel-based files. They're
right. This makes your PDF reprint compact and suitable for printing at any
resolution. However, those considerations are irrelevant at the review stage.
For review, it is best to provide the figures inside your manuscript text document,
usually Microsoft Word, more or less as they would be placed in the final publication.
This makes it easier for the reviewers to read your paper, which is very important.
Reviewers don't care whether you have vector or pixel graphics, provided the
resolution is high enough (about 150 dpi) for everything to look sharp. That's good because
Microsoft Word makes it very hard to achieve vector graphics within your Word
document. Word will import many vector formats, including PDF, but stubbornly displays only a blurry pixelated rendition. The best solution I've found is
to open the PDF in Adobe Illustrator and export it as a high-resolution pixelated
PNG image (e.g. 150 dpi). Word imports and displays PNG images faithfully.
This results in a Word document with sharp pixelated images, perfect for reviewers.
When your paper is accepted, the journal will ask you to provide vector graphic
files for production, one file per image. You can send them your PDF files.
Microsoft Excel produces ugly graphs. I don't recommend Excel, but many students use it because their data are already in Excel, so it's very handy. In that case, bear in mind that the defaults in Excel are contrary to good practice in scientific graphs. Turn off all grid lines. Turn off all the shading and shadow effects.
Kaleidagraph is my favorite graphing program. It's available for both Mac OS X and Windows. It has evolved very slowly in recent years, but is still better than anything else I've tried. I'd love to find a program that's better than Kaleidagraph. The main limitation that I encounter is that the built-in symbols and colors are ugly and poorly chosen. I wish one could use an arbitrary font to provide the symbols.
Kaleidagraph and Word. It's hard to get a graph from Kaleidagraph into
Microsoft Word and have it look sharp when you print your Word document. Here
are two approaches that I recommend: the first is quick and dirty (not sharp);
the second takes many more steps but makes graphs in Word as sharp
as you like.
Copy & paste. The quickest way to get your graph from Kaleidagraph
into Word is to use the Kaleidagraph “Edit:Copy graph” menu item
and then paste that into your Word document. This works, and may be acceptable
for early drafts shared between authors, but the image is usually a bit blurry,
not the crisp sharp image you'd like your reviewers to see. Here's how to get a sharp image into Word:
Print to PDF. Ask Kaleidagraph to print to PDF. The PDF file is postscript, a vector
graphic, crisp. Great.
Crop the PDF. Print to PDF produces an 8.5x11 image. If you're going
to insert this image into a document, it's nice to crop the file down to just
your image. Open your PDF file in Adobe Acrobat Pro. Select “Document:Crop
pages” to get a cropping panel. Enable the radio button “Remove White
Margins”;. Click Ok. Close the document; click Ok to save changes.
Don't insert PDF into Word. In Word, you can select the menu
file" to import the PDF. However, even though the PDF file is a vector
graphic, Word mulishly renders it as a blurry pixelated image. Yuck. Don't
Convert PDF to PNG. Use Adobe Illustrator to open your PDF file.
Use Illustrator File:Export:PNG and set a high resolution (600 dpi) so the
image will later look sharp in Word. The exported PNG file is pixelated, but
has enough resolution to look sharp. (Avoid the export option called “Save
for Microsoft Office”, which inexplicably gives you about half the resolution
you want, so everything looks unpleasantly soft.)
Insert PNG into Word. In Word, select the menu item “Insert:Picture:From
file” to import the PNG file. Word renders PNG images faithfully, so a
high-resolution image will be sharp.
Error bars. Usually every plotted point representing a measured value
should have error bars designating a 95% confidence interval. Usually that
corresponds to ±1 standard error. Please omit the hats that many plotting
programs add to the end of each error bar. The hat adds clutter, making it
harder to see the data.
Error bars in Excel. http://peltiertech.com/custom-error-bars-in-excel-charts
Caption. Every figure should have a caption, beneath the figure, explaining
what’s in the figure. Usually the caption begins with a figure number,
a title, and a description of the horizontal and vertical scales. I prefer
the ordinary English words “horizontal” and “vertical” over
the jargon names “X” and “Y”, or “absissa” and “ordinate”. If your figure includes
someone else’s data or is based on their figure, cite the source at the
end of your caption.
Figures. All text within a figure should be in a sans serif font (e.g. Helvetica, Arial, or Calibri),
including the axis labels, etc. (The figure caption should be in 10 point
Helvetica, to help distinguish it from the rest of your text.) Don’t
use bold. Capitalize only the first letter, as in a sentence, e.g. “Spatial
In general, remember that the graph is meant to express the data, and that
the data themselves should draw the most attention, like the bride at a wedding.
The rest of the stuff (scales, labels, legends) should recede into the background,
not compete for attention. I usually like to represent data as points and
the model as a solid line. We usually use Kaleidagraph (available
for Mac and Windows). We typically use logarithmic scaling. It’s not
an absolute rule, but I find that papers are easier to follow if a log unit
has a consistent length (e.g. 1 inch) within a graph (horizontal vs. vertical)
and throughout the paper. In Kaleidagraph, you set the length of the X and
Y axes by selecting Plot:Set Plot Size:Axis size:. If you do display error
bars, omit the distracting hats at the ends of the error bars. When you send
the final graphics file to the publisher, vector graphics are preferable to
pixelated images, because they look better and take up less space in the final
PDF file for your published paper. Kaleidagraph’s export options are
poor, but you can copy to clipboard a PICT with Postscript, and, if you’re
using Mac OS X or have Acrobat installed, then you can Print to PDF. The latter
is the best way to produce files for production of your article by the publisher.
When you email any graphics file, the filename extension matters: .jpg, .gif,
.pdf. For any other file I suggest that you enclose the files in an archive
(e.g. zip or stuffit) to protect the file resources, which will otherwise be
stripped in the course of emailing.
Symbol size. Some graphics programs, including Kaleidagraph, have poorly
matched sizes of symbols. Obviously, a square and a diamond (the square rotated
45 deg) should have the same area to match visually, but Kaleidagraph matches
them in width, so the diamond is too small for a visual match. Look at your
symbols and adjust the sizes to achieve a visual match.
Color. Some figures, e.g. equiluminant stimuli, demand color. Some
figures may benefit from color, but don’t really need it. Most of your readers don’t
have a color
printer, so, if possible, design your figures to be completely understandable
in a black and white print out. Distinguish symbols and lines by shape and
dashing, and refer to them, in the text, by those achromatic properties.
Helvetica. As of 2007, none of the available versions of Helvetica
are adequate for scientific use. The Helvetica provided by Apple with Mac OS
X lacks italics. (You need italic to correctly represent
mathematical variables, e.g. x. Some programs, e.g. Word, will let you fake an italic,
by slanting, but Adobe Illustrator won’t. ) The versions of Helvetica
sold by Adobe and Linotype lack unicode support. Unicode extends ASCII to a
16-bit code, allowing us to specify any character shape (glyph) independent
of the font. For scientific text this is very helpful because it allows you
to include Greek symbols without changing font. However, unicode is still quite
new and neither Adobe’s nor Linotype’s Helvetica supports unicode.
Helvetica does, but the lack of italics rules it out. Use Arial instead.
Arial. The full history is complicated, but, in effect, Microsoft created Arial by copying Helvetica, to save money. The only difference
I notice is that the “1” in Arial has a longer diagonal line. Type
designers notice other subtle differences.
can test your
ability to distinguish them. (Thanks to Hannes Famira for these links.) The
version of Arial provided by Apple in Mac OS X lacks unicode support. However,
Microsoft Office comes with a better version of Arial, which does support unicode
and includes the Greek characters. So, on your Mac OS X computer, you should
and replace it with a copy of:
Applications/Microsoft Office 2004/Office/Fonts/Arial
Heading and labels. With others, I helped convince Journal of Vision to adopt sentence-style capitalization for all headings and figure labels. Capitalize only the first letter of the first word.
Numbers. When presenting numbers smaller than one, the
decimal point should always be “covered,” so you should replace “.1” by “0.1”.
The problem is that the printed decimal point may be so tiny as to disappear.
If you put a leading zero in front of it, the reader will still know that
Units. Physical measures, e.g. “10 ms”, should always
include a space between the number and the the word(s) specifying the units.
Equations. Don’t confuse mathematics (equations) with computer programming (assignment statements); they have different rules. Here are my suggestions for equations. Use MathType (available for Mac and Windows). Math variables, like E and x,
should be italic and only one letter long. Be friendly to your variables,
don’t set them off by commas. Don’t use “*” to mean multiply.
Don’t use multi-letter variable names; long names are common in computer
programming but confusing in math, where multiplication is implicit,
as in ax. However, you can use a long text subscript, as in Lbackground.
Subscripts that are not variables should not be in italic, eg crms
Functions, like sin and log, are not variables and should not be italic.
Avoid the temptation of indicating an approximate value by “~”, as most journals print that symbol almost indistinguishably from the minus sign “ ”, which is likely to confuse your readers. Instead of “~” use “about”, “roughly”, “approximately”,
or the approximately equal symbol “” that
has two wiggly lines, not one.
Don’t underline. Reserve that for links. Underlining is a proof reader’s mark indicating italic,
which was incorporated into typewriters because they couldn’t do italics.
It was not meant to appear in printed material. Some journals allow it, but
I think it looks bad. In any event, it has become a fairly standard way of
indicating a URL (a web link), so I suggest restricting it to that role.
I also suggest never using the underscore character “_”, especially
in filenames, because if you make that text a link then it will be underlined,
and, once underlined, an underscore is indistinguishable from a space.
Technical tip from Kaleidagraph support: Using font characters as markers. You
can’t modify the markers that are built into KaleidaGraph, but you can
add text error bars to your plot. Text error bars are normally used to annotate
the points in the plot. You could create a Line plot, hide the original markers,
and add text error bars to the plot. As an example, you might create a text
column with the letter a in each row. Once you add this column as text error
bars, you can double-click the text and change the font to Zapf Dingbats or
some other font that has different symbols in it. The manual and help file
contain information on adding text error bars to a plot. To use them as markers,
you would want to add them as X error bars. You would also want to make them
single-sided and have the text centered (using the Center Text option).
Avoid common mistakes
1. Don’t apologize for good work. In
my experience, it is common for students writing their first scientific paper
to end their discussion with a devastating self critique, pointing out that,
with enough hindsight, skill, money, and time, everything could have been done
better to reach a stronger conclusion. So what? Usually this whole self-scourging
paragraph should be deleted. Research has two kinds of limitation—you
could have done better and your conclusions are qualified—and only the
latter should be reported in the scientific paper. (And that report of qualifications
should be brief, just enough to let the reader know.) The reader needs to know
the qualifications of the conclusions of the actual study, but this need is
not served by knowing what the experimenter could have done differently. What
matters is what was actually done. Do the results warrant the conclusion? In
a similar way, conclusions are few and worth telling, but the list of things
that you cannot conclude is endless, so skip it. Needless apologies for what
you cannot conclude or could have done better may worry the reader and undermine
her faith in your words, canceling your scientific contribution. Self criticism
is rare in scientific papers, for good reason. There is a huge supply of manuscripts.
Reviewers are hard-pressed to keep up, trying to separate the wheat from the
chaff. Reading that the author believes the work to be deficient may convince
the reviewer that this manuscript is less worth reading than others and should
2. State limitations laconically. Make
your conclusion as strong as possible, and no more. Its limits must be clear,
but harping on those limits will dismay the reader, shaking her confidence.
Everyone knows that increasing the sample would reduce the standard error.
That goes without saying. As authors, we know best what we actually measure.
Our extrapolations beyond that, to other populations and conditions, depend
on linking assumptions. This is essential to every scientific study and rarely
3. Former intentions don't matter. Sometimes
experimenters measure one thing, thinking that they are measuring another.
Even without mistakes, experimental results often provide a more compelling
answer to a new question, different from the original motivating question.
In any case, the experimenter's former intention is irrelevant to the scientific
report of the results. What matters is what happened, regardless of what the
author was thinking. In scientific papers, the actual history of the author's
thoughts is usually suppressed in favor of a fictional history that streamlines
the argument leading to the final conclusion. In motivating the reader, use
your best current understanding, unconstrained by your original intentions
and motives. The point of the paper is to convince, not to recount. It's science,
4. Significance versus statistical
significance. An undergraduate student
asked me, “If a study does not produce significant results, why would
it be published?” To answer that, we must distinguish two kinds of significance.
The ordinary sense of “significant” (or “insignificant”)
is that something is meaningful and important (or not). “Statistically
significant” (or “statistically insignificant”) means that
we can reject the null hypothesis (or not). These are different things. Showing
statistical significance does not establish importance. Furthermore, showing
that an effect is statistically insignificant can be important.
Usually we only care about big effects. If twice the standard error is small,
a statistically insignificant effect is either small or absent. It’s
not big. Concluding that the effect is negligible can be an important finding.
5. t-test. Statistical significance
is often assessed by a t-test, i.e. whether the sample means differ by more
than two standard errors. I’m looking for a short tutorial that would
explain this to students who may know standard deviation, but are unfamiliar
with standard error. Here’s a link to wikipedia, but it seems longer
and harder than it needs to be. I’m looking for something better. Suggestions?
6. Power: How big should the sample be? I was surprised
when this issue arose in the essays of my 2010 undergraduate seminar “Experiments
in Beauty”. Here are quotes from five of the ten essays (mostly
two-authored). All make a similar comment about the sample size:
study consisted of an unusually small sample size relative to standard psychology
experiments. Statistical analyses are generally intended for sample sizes
of at least thirty participants, thus a greater population may have produced
It is true that our in-class experiments used around 8 observers per
group, much less than the 20 or 30 that is typical of many psychology studies.
However, all of these comments are wrong in presenting this as a flaw. We need
a sample that is big enough to yield a strong conclusion. Statisticians call
this “power”. We need enough samples to attain enough statistical
power to decisively answer the question that we pose. Testing more participants than we need would be a waste of effort, needlessly discouraging us from addressing new questions. In our class, over the semester, we did eleven experiments, each in an hour. There’s no time to waste. We usually
compare the sample means of two groups exposed to different conditions. When
the two sample means differ by at least two standard errors, we reject the
null hypothesis that there is no difference between conditions. The only reason to increase
the sample size is to reduce the standard error, increasing the power. There
is no other benefit. Some of the experiments yielded effects that were statistically
significant: a difference between sample means of at least 2 standard errors.
Other experiments yielded differences that were less than two standard errors,
statistically insignificant. Both results are useful. With more samples the
standard error would be reduced and a small effect that formerly failed to
reach significance might become significant. However, never forget that achieving
statistical significance does not establish importance. No study can ever say
that there is no effect. It can only put an upper bound on how big the effect
might be, namely no bigger than two standard errors (with 95% confidence).
Increasing the sample size in our experiments would not change that basic fact.
It would merely reduce the standard error. In most of our experiments, the
observers were rating beauty on a scale of 0 to 10 and, with about 8 observers
per group, the standard error was about 0.5 beauty points (on that 10-point
scale). To be statistically insignificant, the effect must be smaller than
two standard errors, i.e. less than 1 beauty point in most of our experiments.
In this class we were looking for and found big effects: 1 or 2 beauty points.
We don’t care about small effects,
less than 1 beauty point. The statistically insignificant effects were trivial,
smaller than 1 beauty point, practically no effect. That’s a strong conclusion.
Increasing the sample size would reduce the standard error, but we’ve
already ruled out the possibility of a big effect, so there is no reason to
run a bigger study.
b. “If our study were to be replicated, ... First, we
would obtain an adequate representative sample size of at least twenty participants
c. “The results of our study are limited, ... We
... only had ... eight subjects per group, which is a small sample from which
to make generalizations.”
d. “Another possible future research project
would be to perform this original experiment on a larger sample, ... With
a larger sample size, one can make more broad and general conclusions about
the original hypothesis of this study.”
e. “It is crucial to test a
larger pool of participants ...”
7. What population are we talking about? This
too arose as an issue in the final essays of my 2010 “Experiments in
Beauty” seminar. Three of the ten essays note that their sample is unrepresentative
of the US population:
a. “In addition, our subjects constituted an unrepresentative
sample; the majority of subjects were female psychology students at New York
b. “... a stronger result may be determined by replicating
this experiment with various changes. First, the subject pool in this experiment
consisted of predominantly female NYU undergraduate psychology students. This
sample of students may not necessarily generalize to the entire population.
It is crucial to test a larger pool of participants, ...”
research study comes with strengths and drawbacks. ... Since the participants
were all psychology students with interest in beauty, that may have contributed
to skewed results as they do not accurately represent the general population.
... Possible directions that future research studies can take would be to try
and have a larger sample size that is representative of all people across America.”
These admissions of limitation all make the same implicit assumption. They set the goal
to be conclusions about the general US population. Our class is, indeed, unrepresentative
of the US. But that’s an absurd goal. The National Institutes of Health, supported
by US taxes, is obligated to improve the health of all Americans and encourages
its grantees to follow suit, but, in this class, we have no such obligation.
We can study any group we like, and, for practical reasons, it behooves us
to study the class itself. Having formed strong conclusions about them, we can speculatively
generalize to larger populations, e.g. female psychology majors at NYU,
US undergraduates, beauty scholars, or the whole human race. It’s perfectly fine to study just a small group. Scientific
conclusions about even just one individual are of great value. Much of neurology
is based on case studies, careful study of a single individual leading to strong
conclusions about that person.
Finding articles for your research project
Google Scholar. You are probably looking for articles for your research project. There are various ways to look. Google Scholar is my favorite. Type "scholar" into google, and then select "Google scholar". (Or use the link below.) It's a very powerful searcher of all published articles. You may already know of an article, perhaps from the reference list of another article. You can type in part of the reference and ask Google scholar to find it.
Citation. One very nice feature of Google Scholar is the "Cite" button at the end of the article description. The Cite button gives you text, which is a properly formatted citation for your article, in any of three styles. Select APA, and copy that formatted citation to the Reference section of your paper.
NYU's Get-it. When you find an article, through Google Scholar or otherwise, you'll typically get a link to the journal. Some journals will give you the PDF of the article for free. Some want to charge you, typically around $25 per article. That's a lot of money. However, NYU has an extensive collection of paid electronic subscriptions that allow you to get articles for free. To benefit from these licenses, you must go through NYU. You do this by using the NYU Get-It page. You type some citation information for your article into that page and hit Search. It's particularly easy if you know the DOI number for your article. Then you type in (or cut and paste) just that number. Then hit Search and Get-It will find your article and offer you one or more links to get it, free. Typically you'll be asked to provide your nyu id and password. When it works, you'll get to a journal page that allows you to freely download a PDF of your article.
Fair-use doctrine in US copyright law
Most colleges and universities have online guidelines recognizing that the use of images in journals and theses is protected by the fair-use doctrine. Copyright law grants an exception for reproducing a small portion (e.g. a thumbnail) of a copyrighted work for "fair use". There is a four-part test for fair use, and scientific articles satisfy all four parts, especially if the journal is free:
The law does not specify how small the thumbnail must be to constitute fair use, so the courts have stepped in. Universities and other authoritative institutions have interpreted the court rulings for academics like us. In light of that Amazon uses 100x125 or 125x100 pixels, Google uses 150x150, and the Associated Museum Directors recommend 250x300 pixels. The first document, from Ball State U., is short and easy to read.
Ball State University
Code of Best Practices for Fair Use in the Creation and Curation of Artworks and Scholarly Publishing in the Visual Arts.
Associated Art Museum Directors POLICY ON THE USE OF “THUMBNAIL” DIGITAL IMAGES IN MUSEUM ONLINE INITIATIVES
Get an article:
The Oxford English Dictionary
People with an NYU (or some other university) email address can download Microsoft Office (including Word, Excel, and Powerpoint) for free:
July 29, 2015. Cosmetic.
April 27, 2015. Added link for members of NYU to download Microsoft Office.
January 13, 2015. Added: Images section. Added Fair use section.
January 12, 2015. Added: 0.5. Consent form. Added 11. The paper is not about having guessed right. Renumbered.
December 8, 2014. Added: Figures in place. Number conditions.
November 30, 2014. Added 0. Having an idea. Added 16A. "Done."
October 30, 2014. Added Reference formatting. Expanded Limitations of this study and Future research.
July 27, 2014. Tips for Microsoft Word.
April 29, 2013. Added Finding Articles for your Research Project
May 15, 2011. Curled quotes.
February 6, 2011. Added Read it aloud.
January 31, 2011. Enhanced Title to distinguish being motivated by from mere liking.
January 22, 2011. Added “What do I know after reading your paper that I didn't know before?” Added Reviews. Enhanced Take credit. Specify exactly what is the contribution of this paper.
June 30, 2010. Added Show off your data, not the junk.
June 7, 2010. Added Be complete and Take credit.
May 25, 2010. Added Nudging.
January 11, 2010. Enhanced Convince; Keep going; and Communicate. Added Be friendly; Look at your figures; What is the paper really about?;
Help the reader cite you; and Authors.
Added Avoid common mistakes.
Thanks to Melanie Ceder, Angel Patel, Aretha Soderstrom, Diana Balmori, and
Cesar Pelli for helpful suggestions. Thanks to Mary Mulligan for advice on the fair-use doctrine of US copyright law. I am responsible for what is written here.
to the top