Spotting Bad Science

Andy Brunning has a chemistry-oriented website, but he had a broader-interest post last year with the “Rough Guide to Spotting Bad Science.” It floated around Facebook for a while last summer.

Spotting-Bad-Science

When I teach 100-level classes here at Michigan, and sometimes when I teach higher-level courses, I often include a day introducing the basics of good versus bad science. As the post linked above says, most people get their science from other people or media reports. My reading of this guide is that it focuses mainly on conducting the science and the write-up in the obscure journal (like JGR Space Physics). It includes just a few points (the first two?) related to the equally important issue of “bad science” at the later steps in the process: the media presentation of the research and the individual’s interpretation and usage of that media report. That topic deserves its own post. Here, we’ll stick to the bad science at the researcher level.

What are some key elements for our field from this list? I’ll highlight a few. Correlation and causation (#4) is sometimes an issue for space science as we look for meaning between quantities that don’t have a plausible physical connection. Sample size too small (#6) is another that can easily get us because we are often limited by the observations available or the computing constraints of running a big code. I hope we are not plagued by “Cherry-picked” results (#10), but the temptation is always there to exclude results that do not support a preferred hypothesis. Probably the biggest for us is Unreplicable results (#11). First off, we rely on nature to conduct the experiments and we get whatever the data we can from whatever observatories are available at that time. Second, and more to the point here, is the issue of “closed” data sets, data processing techniques, and computer code. For others to independently verify the findings and build consensus around a hypothesis, everything that went into that result should be available for others to use.

Ethics for Authors

AGU has policies and guidelines covering many aspects of the scientific process. Today I’d like to tell you about one of these: ethical guidelines for authors. The main page on this topic is here, a page which contains a link to the larger PDF document on professional ethics and the broader-scope ethics policy of AGU.

The main point of the page is to give the Top 12 list of ethics expectations for authors of papers in AGU journals. Here are the highlights (my paraphrases of the full text):

  1. Give an accurate account of the research and its significance
  2. Give enough information for others to repeat the work
  3. Cite prior work that’s essential for understanding the investigation
  4. Be complete in documenting the methodology, including assumptions and uncertainties
  5. Follow the appropriate laws governing ethics of work with human or animal subjects
  6. Always provide appropriate citation instead of plagiarizing
  7. Keep the paper focused by avoid the LPU (Least Publishable Unit)
  8. Personal criticism is unacceptable
  9. Report to the Editor any changes made after acceptance
  10. The coauthor list should include everyone that contributed to the study, but only those that contributed; all coauthors share responsibility for the quality and integrity of the work
  11. Reveal to the Editor any potential conflicts of interest regarding authors and list all funding sources in the acknowledgments
  12. The corresponding author should ensure that all coauthors are aware of and approve of the submission (and revision submissions)

Okay, so maybe point 5 isn’t fully relevant to space physics research, but the rest of them are definitely applicable to our field. Please follow them when submitting any manuscript, whether to JGR Space Physics or to another journal.

The webpage above also has a section on “Avoiding Plagiarism.” This includes copying text verbatim out of your own published papers. A sentence here or there is not a problem, but big chunks of paragraphs are unacceptable. Your paper will be sent back to you to have these section rewritten.

calvin_ethics

JGR Space Physics App Features

There are a couple of new features that I have noticed on the JGR Space Physics app on my phone. Maybe these were there all along, but I didn’t notice them when I first wrote about the app. I have been using the mobile app a little more in recent times and I really like what it has, so I thought I would share.

JGRSpaceApp_March2015_v3

One is the button in the lower left corner when you are reading a paper. It opens a menu down the left-hand side with a bunch of shortcut buttons to info about the article. For instance, there is a “Find” button to search through the paper, quick links to the figures and references, a place to grab the citation, and a link to email a link to the article to someone. You can also download a PDF of the paper to your device.

At the Table of Contents for each issue, there is now a new button right near the top that says, “Download Issue.” Yes, a JGR-Space issue is huge (a couple hundred MBs), which is perhaps a little too much for my old phone, but what a cool feature! You can now grab the entire issue in one click and peruse it offline.

Just to say it again, the app is great because once you establish roaming access, then you can read articles anywhere with the app. The instructions on how to set of roaming access are available when you set up the app, which you should do while connected to your subscribing institutions wifi. Remember the critical issue about accessing the app content when away from your subscribing institution: you have to reinstate/refresh roaming access every 90 days. This is a bit of a pain but is the tradeoff to prevent fraud and abuse of access. From your subscribing institution’s network, you have to go to http://onlinelibrary.wiley.com, click the “log in/register” button in the upper right corner, then click the “My Profile” button in the upper right part of the page, then click “Roaming Access” in the left column, and finally click the link that says “Refresh Roaming Access” (if it is still active) or “Reinstate Roaming Access” (if it has expired). This allows you to keep going with anywhere-access with the app for another 90 days. Perhaps you can be space-oriented about remembering to do this and refresh every solstice and equinox.

Yes, that’s a lot of steps to renew “anywhere” access and yes, but it’s worth it when you are waiting to pick up your daughter from track practice. Doesn’t everyone read JGR Space Physics while waiting around for something?

Disputing a Rejection

Over the more than year of being Editor-in-Chief, I have had several authors contact me about their paper that was rejected. These authors want to argue the case that the rejection was not justified and are asking that me (or another Editor) reverse or at least reconsider the decision. Luckily, these emails are nearly always polite and respectful; you are a great crew of people that know how to be tactful and considerate. Thank you!

Quick-writing-makeovers

Let me reassure all of you about your rejected manuscript: if you submit a well-reasoned rebuttal to the points made in the reviews, then your paper will always be thoughtfully reconsidered by the JGR Space Physics editorial board. If you put in the time to write a response and appropriately revise the manuscript, then we will carefully and thoroughly read that response and examine those changes to the paper. You always have the option of resubmitting a rejected paper back to JGR Space Physics.

Our “full consideration” of the manuscript might lead to several outcomes. We might decide that it is still not appropriate or ready for the journal and reject it again without even sending it out for review. More likely, we will send it out for review to the same set of referees. We’ll put a special note in the review request letter to note that this is a resubmission of a rejected paper and that we would very much appreciate their assessment of the new version. Furthermore, depending on the nature of the rebuttal you have provided or the scientific controversy being debated in the paper, we will solicit additional referees for it. When a paper is in this situation, we like to have a consensus view about it, so we will send it out to more than just the standard two referees.

If we decide to reject the manuscript again, then we will give you a full explanation of our position about the paper. Even after a second (or nth) rejection, you still have the option of resubmitting to JGR Space Physics. However, at this point, arguing your position is a harder task and it is a far better decision to choose to revise the manuscript according to the referee or editorial suggestions and concerns.

I have found, though, that occasionally it was a misunderstanding on the part of the referees and/or me about the content of the paper. In these cases, the resubmitted paper sails through review and is accepted within a round or two. Yes, I have seen papers accepted at “round 1″ of reviewing; these are invariably resubmissions of rejected papers where the authors either made all requested changes or offered such a convincing counterargument that the reviewers (and me) were convinced.

Everyone is frustrated when they get a rejection letter from a journal editor. Here at JGR Space Physics, we have high standards for the journal that we are trying to upload and we implicitly trust our reviewers to do a thorough and competent assessment. After that initial emotion passes, though, think carefully about the comments we provide and, if you think resubmission is warranted, then please feel encouraged to do so.

Revised Reviewer Instructions

To continue with the changes at GEMS, the reviewer instructions are also a bit different than they have been in the past. At the GEMS login page, there is a link across the top called “Reviewer Instructions.” I encourage everyone, no matter how experienced you are with reviewing papers for JGR Space Physics, to visit this page and get up to speed on the latest reviewing criteria, guidelines, and tips.

To quickly summarize, referees are now asked to answer four questions with drop-down menu options. They are:

  • Is the paper significant and convincing?
  • Do the methods and analysis support the conclusions?
  • Is the referencing appropriate?
  • Is the presentation high quality?

For each of these questions, there are several possible answers ranging from a strong “yes” to a definitive “no.” You will also be asked for your overall assessment and recommendation regarding the paper (from “publish as is” to “reject”). There are several other questions you also need to answer about whether the paper is worthy of highlight in the Research Spotlight section of Eos or if a figure in it is worthy of the Image Carousel at the top of the journal home page.

After this, you will see two comment boxes for uploading your formal review. You can still upload a file, if you so choose, rather than cutting and pasting into the text box. The Reviewer Instruction page lists a series of questions that we would really like you to address in your formal review. In addition (or as part of these answers), please elaborate on your answers to the four questions listed above, whether complimentary or critical, in your formal review.

We really appreciate your time and effort to referee papers for JGR Space Physics. We love to get thorough and thoughtful reviews and they help us tremendously in assessing the paper and making a decision on publication. Furthermore, we really like to get details about what you liked in the paper. While the criticisms and suggestions for improvement are essential, pointing out the good aspects of the paper and providing supporting text to defend that appraisal is also very useful to us.

Awesome Author Instructions

AGU has been making changes to the GEMS system over the last few months, and I’d like to let you know about one thing that I find very cool: the author instructions page. At the GEMS login site, there are several links along the top, and one is Author Instructions. This page gives you lots of information detailing what you need to know for submitting a manuscript to JGR Space Physics (or whatever AGU journal, this is available on all of the journal GEMS pages).

The “links” on the left expand into text right there on the page: “Journal Specific Requirements” discusses publication fees and paper types; “Initial Submissions” lists all of the information and files you need for the first upload of a paper; “Resubmissions and Revisions” tells you what you need for the subsequent uploads, which are slightly different because they want to streamline the transition to publication; and “Submission Process” gives you a heads-up on the order and format of the steps to submission.

On the right are links to other pages with information about the journal and some more detailed information about formats, guidelines, requirements, and available templates. While I like this entire page and think that AGU did a fantastic job laying out for easy use, the coolest thing on this page, in my opinion, is the Submission Checklist. This downloads a PDF with the full listing of all manuscript parts, submission steps, and file requirements. Perhaps its just me because I am huge fan of checklists, but this distills the entire process into a single sheet. Furthermore, if you need more information, the checklist includes many hyperlinks to the original documents with the full details on that topic.

AGUauthorsubmissionchecklist_header

Okay, I might be a little weird to be so excited about author instructions, but I find this page, and especially the checklist, to be a one-stop-shopping treasure trove of information that I hope makes your lives as JGR Space Physics authors much easier.

Come to the TESS Meeting

The Triennial Earth-Sun Summit, or TESS, is rapidly approaching. This is a new meeting for the space physics community: a joint venture between the Space Physics and Aeronomy section of AGU and the Solar Physics Division of AAS. I am sure that many people have been involved in planning this new conference, but I know that Jim Klimchuk, a solar physicist who has just served as AGU’s SPA President, was instrumental in bringing the leadership of the two communities together to make this meeting happen. I would like to strongly encourage everyone in space physics to attend TESS this year and help make it a great meeting. It’s not too late to decide to go, either: the abstract submission deadline was shifted to February 24, so you still have time!

TESS-web_header_v3

There are a lot of common physical processes between solar research fields and those of the rest of the SPA scope of subjects: plasma physics, the transition from collisional to collisionless transport, ion-neutral interactions, energetic particle acceleration, magnetic reconnection, wave excitation and wave-particle interactions, and the relationship between plasma, currents, and magnetic fields, just to name a few. Not to mention the traditional “one-way” interactions of solar EUV/X-ray impacts on planetary upper atmospheres and solar wind influences on planetary magnetospheres and ionospheres; a meeting like this can help Earth and planetary scientists better understand the origins of these driving phenomena on the systems they study.

To relate it to JGR Space Physics, I really hope that this meeting spawns new collaborative investigations and eventually papers submitted to the journal. I think that this meeting will be a fantastic opportunity for cross-disciplinary discussions that hopefully will lead to new insights and research initiatives. I am greatly looking forward to this meeting, not only for myself as a researcher but also in my role as Editor-in-Chief. I will have my eye out for special section opportunities, but if you would like to suggest one as a follow-up to this (or any other) meeting, then please feel free. The form is here.

As the “triennial” in the name implies, the SPA and SPD leadership would like this meeting to become a regular every-third-year event on our schedule. However, I think that “Earth-Sun” is a little limiting…I don’t think that the plan was to exclude planetary space environment scientists from the exchange, and in fact a number of the invited speakers are planetary experts who will almost certainly discuss the connection to planets other than Earth. So, I would like to make a special call to planetary scientists to consider attending TESS: please feel welcome to submit at abstract.

Note that TESS is organized a little differently than a “normal” AGU meeting. With this first one being managed by AAS, they are largely following their format of not scheduling special sessions but rather having “open” abstract submissions. The conference organizers will sort the submissions into sessions and produce a meeting plan based on what people will talk about. I think it will work out just fine and allows a lot of flexibility to organize cross-cutting sessions that hopefully serve the purpose of bringing the communities together.

Finally, I’d like to make a plug for the location. Indianapolis is a enjoyable city with a great walking downtown area with lots of restaurants, shopping, and local attractions. I visit Indy occasionally and I like it a lot. I hope that you do too. SPD apparently had a meeting there recently and liked it so much that they suggested it for the inaugural TESS conference. Good choice!

Editing While Ill

Oof da, this has been a tough week. I don’t get sick very often, and when I do it usually passes pretty quickly, but this cold has lingered all week long. Most days I come home and crash, sleeping for 10 to 12 hours, and still not feeling very rested. I have managed to keep up with the essentials at work: keeping my classes going (yes, plural); attending and leading a few committee meetings; meeting with students; getting a few letters of recommendation written; and assigning editors to the new manuscripts submitted to JGR Space Physics.

The rest of it, though, has suffered greatly this week. This includes a slowdown of some of my editorial duties. I haven’t let it all slide; I have been occasionally plugging away at it and keeping the volume under control (sort of), but my red arrows in GEMS are more numerous this week than usual because I haven’t been able to devote as many hours to editing as I usually do. I’m feeling better today and I hope to catch up.

Especially ignored this week has been email. I apologize to all I have offended with my silence or delayed response. It’s not you, it’s whatever virus is putting up a fight in my body and taking away all of my energy and ability to concentrate.

So, like I have written before, email correspondence is often not my highest priority and sometimes, when life limits my work hours (like this week), unanswered emails back up in my inbox to embarrassingly high levels. Please be patient and feel free to resend something to me. I am not annoyed by reminders…on the contrary, I appreciate the fact that you care enough to politely prod with a follow-up. They elevate your note to page one of my inbox and I will hopefully respond to it quickly.

There is no image with this blog post…I don’t think you’d want one.

So, like I have said before (link: late June post on emails), email correspondence is often not my highest priority and sometimes, when life limits my work hours (like this week), unanswered emails back up in my inbox to embarrassingly high levels. Please be patient and feel free to resend something to me. I am not annoyed by reminders…on the contrary, I appreciate the fact that you care enough to politely prod with a follow-up. They elevate your note to page one of my inbox and I will hopefully respond to it quickly.

New Year’s Resolution

Welcome to 2015! It’s going to be a good year.

I am making a New Year’s Resolution: writing every workday. Let me be even more specific: In 2015, I resolve to spend at least 30 minutes each workday on writing scholarly papers.

Most people I know that are willing to admit their scholarly writing habits are “binge writers.” That is, they don’t write every day but rather block out a chunk of time every now and then, perhaps even weekly, for paper writing. In recent years, I fit into this category as well.

I am not an expert in the subject of the most effective method of academic writing but every expert that I have heard on this topic says that binge writing is not particularly effective. While people think that it is more effective to “get in the zone” and spend a whole day writing, research shows that binge writers end up writing significantly fewer pages over the course of a year.

It is much better to write a little bit every day, preferably at the same time and place, to establish a long-lasting habit. The rhythm of daily writing will make it so that getting in the zone is very quick, therefore allowing meaningful progress even with only a few minutes of writing.

Here is a website where you can learn more about the trade-off between binge writing and daily writing:

http://www.academiccoachingandwriting.org/academic-writing/academic-writing-blog/ii-challenge-common-assumptions-against-daily-writing/

So, I am making a resolution to spend at least 30 minutes a day on paper writing. Not proposal writing, email writing, other people’s papers, or blog posts, but first-author papers. Some of the time could be spent outlining a paper; I am one that always starts with a full outline and convert each bullet point into a sentence or paragraph. This time might include reading journal articles relevant to an Introduction section, as long as I take notes in the paper outline and use the time to help fill in at least a few words towards a manuscript. I am not going to count initial plot-making time but it might include a small modification of a plot now and then just to touch it up for a paper. Mostly, though, it will entail writing. I have four partially written manuscripts that I would like to get submitted this year. I hope that I can make it within the first six months of 2015 but that’s not in my resolution. For my resolution, it’s just the writing part.

Power of Words

Please feel free to join me in making a pledge to focus on writing papers in 2015.