Data Set Repositories

AGU’s enactment of an open data policy for all papers in its journals has moved up a notch. The current enforcement of the policy is that “available upon request from the author” is no longer allowed. The data that you use in a paper, on which you are advancing our understanding of the space environment, must be available to others. Remember, “data” is not just observed values but numerically generated values as well.

For many observational data sets, openness is required by the major US funding agencies, NASA and NSF. In fact, even for small grants, they now require data management plans about how the data produced by the project will be stored and made accessible to others. NOAA has a lot of its data freely available through several avenues. If you do simulations at the CCMC, then your run output is available to all at that website. That is, for many things, we can simply list a website and call it good.

The issue is for small data sets, like laboratory experiments or temporary instrument installations, and in-house simulation results. Authors using such data need to make the numbers available to readers without the reader being required to go through the author. Furthermore, the website where the data are available needs to be a permanent and independent repository, not the author’s personal site. We need others to be able to independently check our results, reproduce our plots and tables, and verify our claims.

For those at big institutions, like me, such places are creating open repositories for their researchers. For instance, the University of Michigan has a site called Deep Blue. We are putting data bricks there from specific, published papers.

Many have asked about “public repositories” that will accept a data brick accompanying a journal article. There are several. AGU is associated with COPDESS, the Coalition on Publishing Data in the Earth and Space Sciences, which is an organization that maintains a list of scientific repositories. It is easily searchable and includes heliophysics and space physics as taxonomy groups. One data base listed there for space physics is NCEI, the National Center for Environmental Information, which has this for its data archive submission front page. AGU also recommends general ones like  ZenodoDryad, or Figshare – each can assign a DOI for deposited data. Github is becoming a common place to share not only code but also code output.

Data_Repositories_small

            The AGU Data Policy FAQ page has a lot of good information about current implementation and additional suggestions of repositories willing to host your data.

Another question that I get is, “how much to upload?” My common answer is, “As much as you can.” Seriously, though, some numerical simulations produces hundreds of GB of output, and some statistical surveys of observational data can cover several TB of values. I don’t want to quote all of the policies for every data repository but there are some out there that will take very large data sets. The minimum set should be “those data used in the paper.” This includes the values behind any plot, table, or value in the paper.

Advertisements

New Editors for JGR Space Physics

The editor search for JGR Space Physics is done and the search committee has selected two new editors for the journal: Natalia Ganushkina and Viviane Pierrard. These two are highly qualified for the role and the final decision was quite difficult. We think that they will serve the space physics research community very well. With my amazing photo-editing powers, I have added them to our group picture:

NewEditors_June2018

            Remember that AGU is rapidly approaching its 100-year birthday in 2019, and there are many plans for celebrating this existence milestone. I have appointed one of the JGR Space Physics editors as the coordinator of our Centennial activities – Larry Kepko. So, he has been pulling back from “normal” editing assignments in order to arrange our Grand Challenge paper set and organize a collection of historic perspectives from and about the pioneers of space physics. I think that Dr. Ganushkina will be picking up a lot of this workload of manuscripts on the outer magnetosphere and tail, storm physics, and substorms.

We also receive many submissions on inner magnetospheric topics, especially the radiation belts. Dr. Balikhin and I handle most of these manuscripts, but the volume is large. Because we also cover papers in other disciplines within the journal scope, this is heavy load. In addition, I would like to do more of my editor-in chief duties that sometimes get the short end of my attention, like long-range strategic planning, publications policy discussions, and communication (like paper publicity and this blog). Plus, I am now an editor liaison on the AGU Meetings Committee, which is a very interesting position but takes additional time. I think that Dr. Pierrard will be picking up a lot of the rebalanced workload of inner magnetospheric manuscripts. She will also help us better connect with the solar physics community.

The biggest selection criteria applied by the search committee were expertise in their research field, demonstrated reviewing excellence or editorial experience, and an editorial philosophy that blends well with the existing team. The search committee also took into account geographical, disciplinary, gender, and racial diversity/breadth in their decision. In fact, AGU is making a concerted effort to increase representation of women on its journal editorial boards, and JGR Space Physics was one of only two AGU journals with an all-male editor crew. The search committee happily included this criterion in its deliberations.

Note that these two new editors are being appointed for 4 years, so they will continue to serve after I rotate off late next year, when my term as EiC ends along with the terms of the 4 other editors. This timing is intentional in order to ensure some editorial continuity between EiC terms.

We had many excellent candidates and, I would like to reiterate, it was a very difficult decision to select only two. AGU does not limit the size of our board but the search committee made the downselect to the originally-advertised two positions so that the next EiC has some flexibility in selecting new editors for their team. There are definitely some in the candidate pool that I will be encouraging to apply for the EiC or Editor positions that will open up in a year or so.

Unconscious Bias in Space Physics

I attended the Triennial Earth-Sun Summit meeting a couple of weeks ago, and there was a very good plenary session on unconscious bias in space physics. The presenters were the authors of the Clancy et al. paper in JGR Planets on bias in astronomy and planetary science. They summarized the findings of that paper, which quantified the extent of women and minorities reporting feeling unsafe or encountering a hostile work environment in these science fields. The numbers are not encouraging, with 80% of women experiencing some kind of sexist remark and two-thirds of women-of-color hearing racist remarks in the workplace. Furthermore, over a quarter of women have felt unsafe in their current position because of their gender or race. This is disturbing to me that the numbers are so large in 2018.

Unconcious Bias Plenary Handout title

            Fortunately, the conversation is not ending with the TESS plenary session. The organizers created a handout that was available to everyone at the session and online with the session description. I highly encourage everyone to read this tri-fold pamphlet. They encourage people to take the Harvard implicit bias test and read through the materials at the U of Arizona’s StepUp! by-stander intervention program. The sheet is filled with tips on how to identify and minimize implicit bias. Two of the biggest things that individuals can do immediately: amplify minority voices is group discussions (but don’t he-peat) and avoid making sexual remarks in the work environment.

As for JGR Space Physics, fighting implicit bias can be done in several ways. The first is to be cordial in your correspondence, especially to early career researchers like graduate students, and to apply the Platinum Rule in your interactions with others, thinking about how they want to be treated and considering the interaction from their perspective. Authors, please use gender-neutral pronouns in responses to anonymous reviewers. Reviewers, consider using one of the links in the handout for quantifying gender bias in writing. Finally, I hope that you all make a personal DEI pledge to promote diversity, equity, and inclusion. People leave the field because of sexism in the workplace, and for our discipline, the workplace includes manuscript correspondence. I occasionally hear from advisors whose students have had a bad interaction with a reviewer.

Thanks to the TESS meeting and session organizers and for coordinating this panel discussion. Let’s continue to strive to do better to reduce implicit bias in space physics.

Length of Your Review

A question posed to me recently asked about the best length for a manuscript review. Let me quote from the email:

“On one side of the spectrum is a group of people that will only comment if something is blatantly wrong. On the other side of the spectrum is the reviewer that will give a lengthy response including all of the changes that could improve the study (or at least as improve the study in the eyes of the reviewer).”

Neither of these extremes is optimal. A very short, highly negative review is particularly bad but close behind is a very short, highly positive review. The more details I have, the better that I can assess the manuscript and weigh the recommendations from the two reviewers. So, in general, I like longer reviews. Reviewers can, actually, go too far, suggesting additional studies and analysis that might be very good to conduct but are not necessary for publication of the submitted manuscript. To me, an optimal review includes praise of the good parts of the paper, identification of what is wrong or unclear, and suggestions for what will make the study publishable.

As I said in one of my first posts on this blog, please be thorough – I like the longer reviews better than the short ones. If you need to write several paragraphs to explain why some aspect of the study is off base, then please do it. That helps me make the right decision about the manuscript.

The main place that I find reviewers being too verbose is with suggested new work. If you write two paragraphs on how the authors should really include another section, then please stop and ask yourself: is this new section that I am asking for necessary to make the submitted manuscript acceptable for publication? If not, then your two paragraphs are really a suggestion for future work. Such suggestions are fine but it should be noted as such. If not noted, then Editors and authors assume that it is a recommendation for acceptability, and authors should either do it or write a thorough rebuttal as to why they didn’t do it.

At the AGU reviewer resources page, the text begins with the main goal of peer review:

“AGU Publications relies on our reviewers to help ensure the standards, quality, and significance of our papers.”

Ensuring the standards, quality, and significance of papers in AGU journals is, I think, best served by identifying those elements of the manuscript that make it fall short of being a “significant contribution to the field.” A terse report, positive or negative, is less than fully helpful to me as an Editor. Similarly, it is useful to expand, at least somewhat, on the positive elements of the manuscript that raise it to the level of acceptability.

Down the page a bit is this definition of a good review:

“In general, the most helpful review is one which first provides an overall summary of the main contribution of the paper and its appropriateness for the journal and summarizes what major items should be addressed in revision.”

I agree. A very brief summary of the paper is always appreciated, not only for the Editor but also for the authors, so that they know that you understood the main point of the paper. Positive comments on the appropriateness for the journal are good, too, especially if the other reviewer finds fault with the same aspect of the manuscript. While many reviews include these elements, a large percentage of reviews do not, instead jumping in to the negative comments. Having these two other elements, a neutral paragraph reiterating the main point of the paper and a positive paragraph highlighting what is good about it, also helps to set a cordial tone to the report, which is always a good thing, in my view.

I like the figure in the Eos article about writing a good review, linked on this page.

peerreviewguide-flowchart

It highlights that the first things a reviewer should write are these two elements, the summary and the positive aspects of the paper. Then move on to the concerns and suggestions.

This page also has a link to another page with the lists of questions and pull-down menus that reviewers will be asked as part of the submission of their report. This is not the full list of all things to consider in your review, but these are the questions that AGU would like to see answered about every manuscript sent out for peer review. Explanations of the answers to these questions should also be in the main text of the review.

So, longer reviews are, in general, better for the Editor. Just take this advice with some caution, however, and think twice about suggestions for new work. Make it clear in the report which suggestions are, in your view, simply possibilities for future investigations and which are recommendations for acceptability.

The Moldwin Paper on Citations

Mark Moldwin and I recently published a Commentary on, well, hopefully the title says it all: High-citation papers in space physics: Examination of gender, country, and paper characteristics. He obtained the article information for every paper published in JGR Space Physics in the year 2012, including the citation count as of June 2016 for each paper, and then classified the papers according to, you guessed it, gender, country, and paper characteristics. There were 705 papers in the journal that year, so this task took quite a while to complete, plus we took some time discussing which parameters to even classify for later use. We then analyzed these results to see which qualities about the paper had a statistically significant connection to citation count. A fairly recent year was deliberately chosen to investigate the factors related to citations early in a paper’s lifespan, a time interval of relevance to the calculation of the Journal Impact Factor. As of today, it is still “in press,” so just the accepted version is online, but the paper is Open Access so it is free to read the full text.

MoldwinPaper_header

            Here are the major findings. These qualities of the paper are correlated with more citations in the first few years after publication:

  • More coauthors
  • More institutions in the author affiliations
  • More countries in the author affiliations
  • More references in the paper
  • A colon in the title

These qualities of the paper had no significant correlation with citations:

  • Gender of the first author
  • Number of words in the title
  • Acronyms in the title
  • Geophysical region names in the title

Keep in mind that the standard deviations are wide, so these findings are not necessarily true when comparing any two papers from the “high” and “low” classes. Welch’s t-test statistic uses the standard deviation of the mean, which is a much smaller number than standard deviation (the spread for any one data point in the set), Any individual paper, regardless of its characteristics, could have a high or low citation count a few years after publication. That is, we did not find a “magic parameter” that clearly identifies what will make a paper get many citations, nor one that easily picks out the low-citation papers.

Furthermore, the underlying distribution of values is not Gaussian – but any subset we considered, there is a long, positive tail creating a non-negligible skew to the histogram – yet the probabilities for significance that we used are based on a normal distribution for the two populations. This is why we used a 99% “highly significant” threshold to determine those qualities that are connected to citations.

So, take all of these findings with a grain of salt. Nevertheless, we think the results are interesting for the space physics community to know. The main conclusions that more authors, institutions, countries, and references increases eventual citations are not particularly surprising, but this is the first time it’s been quantified for papers in the field of space physics.

Two results are surprising to us. The first is that there is not a statistical difference in the citation of papers based on the gender of the first author. Other studies have found such bias in other fields, including in other closely related natural sciences, like astronomy. Unlike those studies of other fields, we did not find a statistically significant difference in citations to JGR Space Physics papers based on that parameter.

We did not expect to find any “title parameters” to be connected with citations and most were not. We were rather amused, however, to find that a colon in the title is linked to higher citations. About 20% of the papers that year had a colon in the title. That’s over 100 papers so this is a decently large sample size. We have guesses but, really, we have no good explanation for this. For those wondering, yes, this finding did indeed influence the title of our paper.

In summary, our advice to potential authors of manuscripts for JGR Space Physics is this: collaborate with others and cite the literature. It’s not a guarantee that your paper will receive above-average citations but, based on our analysis, it might help. Happy writing!