DORA and the JIF

No, this isn’t a young Latina’s adventure story about mantequilla de maní (or crema de cacahuate, or one of the other translations for peanut butter). DORA is the Declaration on Research Assessment and is a call to action to put less reliance on the Journal Impact Factor (JIF). This is timely because the new JIFs for 2017 were just released this week.

DORA - logo@2x_withblack

            I particularly like the “general recommendation” of DORA:

  • Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.

It is a simple yet powerful message – the JIF (or any other journal metric) is not a measure of the quality and impact of any individual paper in that journal.

For 2017, the JIF for JGR Space Physics is 2.752, which is ever-so-slightly higher than last year’s JIF. A few notes about it:

  • Remember that the JIF is calculated only from citations in one year to papers published in the previous two years. It is an average of a highly skewed, non-Guassian, positive-definite distribution of a very small subset of full journal content.
  • I like to only quote the JIF to two significant figures, so this year’s value is 2.8, which, due to rounding, appears as a small improvement over last year’s 2.7 value.
  • There are other journal metrics out there and I haven’t yet seen these values for 2017.
  • The JIF for Space Weather is 2.9, the first time that that journal’s JIF is bigger than the JIF for JGR Space Physics. Way to go, Space Weather!
  • AGU’s few-year-old journal Earth and Space Science received its first JIF this year, coming in at 3.2. Awesome job, ESS!
  • The journal had significant growth in terms of papers published from 2015 to 2016, up by 42, which is more than 5%. The Scholarly Kitchen just had a post a couple of weeks ago stating that journal growth lowers JIF. So, the fact that the journals JIF went up means that the citations outpaced the negative impacts of growth.
  • Historically, the citation rate to articles in JGR Space Physics are rather constant with time, so that a ~10 year old paper has ~29 citations, on average. This is just how our research community likes to cite papers and it would take a massive cultural shift to alter this trend.
  • JGR Space Physics historically has a “cited half life” of at or above 10 years, which means that a 10-year-old paper with X citations will, on average, end up with roughly 2X as its eventual total citation count.
  • Nearly all papers in JGR Space Physics receive at least one citation, which is not the case for nearly half of the papers in the vast Web of Science database.

In summary, I think that the journal is doing very well. Thanks for continuing to support it. Finally, while the post was mostly about the new JIF, I’d like to leave it where I started, on the positive future outlook of DORA, in which we put the JIF in proper perspective according to its strengths and weaknesses as a journal metric, and especially stop using it to assess individual research articles or investigators.

Advertisements

New Editors for JGR Space Physics

The editor search for JGR Space Physics is done and the search committee has selected two new editors for the journal: Natalia Ganushkina and Viviane Pierrard. These two are highly qualified for the role and the final decision was quite difficult. We think that they will serve the space physics research community very well. With my amazing photo-editing powers, I have added them to our group picture:

NewEditors_June2018

            Remember that AGU is rapidly approaching its 100-year birthday in 2019, and there are many plans for celebrating this existence milestone. I have appointed one of the JGR Space Physics editors as the coordinator of our Centennial activities – Larry Kepko. So, he has been pulling back from “normal” editing assignments in order to arrange our Grand Challenge paper set and organize a collection of historic perspectives from and about the pioneers of space physics. I think that Dr. Ganushkina will be picking up a lot of this workload of manuscripts on the outer magnetosphere and tail, storm physics, and substorms.

We also receive many submissions on inner magnetospheric topics, especially the radiation belts. Dr. Balikhin and I handle most of these manuscripts, but the volume is large. Because we also cover papers in other disciplines within the journal scope, this is heavy load. In addition, I would like to do more of my editor-in chief duties that sometimes get the short end of my attention, like long-range strategic planning, publications policy discussions, and communication (like paper publicity and this blog). Plus, I am now an editor liaison on the AGU Meetings Committee, which is a very interesting position but takes additional time. I think that Dr. Pierrard will be picking up a lot of the rebalanced workload of inner magnetospheric manuscripts. She will also help us better connect with the solar physics community.

The biggest selection criteria applied by the search committee were expertise in their research field, demonstrated reviewing excellence or editorial experience, and an editorial philosophy that blends well with the existing team. The search committee also took into account geographical, disciplinary, gender, and racial diversity/breadth in their decision. In fact, AGU is making a concerted effort to increase representation of women on its journal editorial boards, and JGR Space Physics was one of only two AGU journals with an all-male editor crew. The search committee happily included this criterion in its deliberations.

Note that these two new editors are being appointed for 4 years, so they will continue to serve after I rotate off late next year, when my term as EiC ends along with the terms of the 4 other editors. This timing is intentional in order to ensure some editorial continuity between EiC terms.

We had many excellent candidates and, I would like to reiterate, it was a very difficult decision to select only two. AGU does not limit the size of our board but the search committee made the downselect to the originally-advertised two positions so that the next EiC has some flexibility in selecting new editors for their team. There are definitely some in the candidate pool that I will be encouraging to apply for the EiC or Editor positions that will open up in a year or so.

Unconscious Bias in Space Physics

I attended the Triennial Earth-Sun Summit meeting a couple of weeks ago, and there was a very good plenary session on unconscious bias in space physics. The presenters were the authors of the Clancy et al. paper in JGR Planets on bias in astronomy and planetary science. They summarized the findings of that paper, which quantified the extent of women and minorities reporting feeling unsafe or encountering a hostile work environment in these science fields. The numbers are not encouraging, with 80% of women experiencing some kind of sexist remark and two-thirds of women-of-color hearing racist remarks in the workplace. Furthermore, over a quarter of women have felt unsafe in their current position because of their gender or race. This is disturbing to me that the numbers are so large in 2018.

Unconcious Bias Plenary Handout title

            Fortunately, the conversation is not ending with the TESS plenary session. The organizers created a handout that was available to everyone at the session and online with the session description. I highly encourage everyone to read this tri-fold pamphlet. They encourage people to take the Harvard implicit bias test and read through the materials at the U of Arizona’s StepUp! by-stander intervention program. The sheet is filled with tips on how to identify and minimize implicit bias. Two of the biggest things that individuals can do immediately: amplify minority voices is group discussions (but don’t he-peat) and avoid making sexual remarks in the work environment.

As for JGR Space Physics, fighting implicit bias can be done in several ways. The first is to be cordial in your correspondence, especially to early career researchers like graduate students, and to apply the Platinum Rule in your interactions with others, thinking about how they want to be treated and considering the interaction from their perspective. Authors, please use gender-neutral pronouns in responses to anonymous reviewers. Reviewers, consider using one of the links in the handout for quantifying gender bias in writing. Finally, I hope that you all make a personal DEI pledge to promote diversity, equity, and inclusion. People leave the field because of sexism in the workplace, and for our discipline, the workplace includes manuscript correspondence. I occasionally hear from advisors whose students have had a bad interaction with a reviewer.

Thanks to the TESS meeting and session organizers and for coordinating this panel discussion. Let’s continue to strive to do better to reduce implicit bias in space physics.

Preprint Servers: Challenges

A third (and probably final, for now) post on of ESSOAr, AGU’s new preprint server for Earth and space sciences. The first described it, the second touted it, and now this one is the ethical scold of how best to use it.

The biggest point to remember is that preprint servers are not peer-reviewed journals. Yes, there is an editorial board that checks submissions for scientific scope, but there is no vetting of the accuracy of the content. The editorial check takes a day or two, maybe a week max, but it is not a real review process. Yes, content here gets a DOI, but we should all remember that content on preprint servers are essentially just a step above “private communication” in terms of referencing authority. That is, it could be wrong.

We hope that content on ESSOAr, and any other preprint server, will eventually be published in a scientific journal. Researchers are putting their reputation out there with each new post on one of these servers, so the content is, for the most part, respectable. Go ahead and use it to learn what is being done by your colleagues. Because preprint server content has not been through the peer review process, though, it should be replaced with the “final” version of the study from whatever journal it eventually appears in.

To summarize in a graphic:

Caution-preprints

            Peer review should still be the standard for what is accepted as “knowledge” of the subject. Even this can be wrong but at least it has been thoroughly scrutinized by experts. You should be very skeptical of older preprints on the server (say, more than 2 years since original posting) that lack a link to a final published version of the paper. That work either was not submitted or did not pass peer review. If the former, then it is perhaps the case that the authors found a problem with the study and therefore never submitted that version of the paper. If the latter, then perhaps the editor or referees found a problem with the study and declined publication of it. Either way, the study did not reach its “final” form in the literature.

The advice to the community about older preprints can be summed up like this:

  • Authors: use caution when citing an older preprint.
  • Reviewers: pay extra attention to citations of older preprints.
  • Editors: ask reviewers to check the appropriateness of older preprint citations.
  • Societies: set policy about citing older preprints.

I am told that the astrophysics community, which regularly uses the arXiv preprint server, understands this difference in “publication” levels. That is, research communities can learn to use preprint servers and make it their go-to place for the latest content across a number of journals, as I am told that many in astrophysics do. They also know, however, that when it comes time to write your own paper, don’t rely on preprints as your main entries in the reference list. The astrophysics community, I am told, understands the guidelines about preprint servers and only uses it for finding the latest work on a topic.

We, the Earth and space science research community, should adopt this same mentality about preprint servers, not only ESSOAr but any server (and there are several being created). Such servers should be a place to get the latest studies from across a variety of journals, learning about content as the manuscripts are submitted rather than months later when they are accepted and eventually published. We should only use it for the latest work, though. A preprint server is not the place for full literature searches – those should be done in Web of Science, Google Scholar, Scopus, ADS, or other services that scan the published, peer-reviewed literature. And, as an editor, I strongly urge you to please conduct a full literature search, because a recent study by Mark Moldwin and me showed that the more complete your reference is, the more citations your paper will get (on average).

Use ESSOAr, but know its purpose within the hierarchy of scientific publications.

Preprint Servers: Benefits

With the launch of ESSOAr, AGU (and all of the other supporting societies on the advisory board) has entered the market of posting scholarly content prior to official acceptance by a peer-reviewed journal. Yesterday I discussed the “how” of ESSOAr, here I discuss the “why.”

The big reason is to increase scientific communication and collaboration. AGU’s mission is to promote discovery in Earth and space sciences, and many of the society’s honors, medals, and awards cite “unselfish cooperation in research” as a primary criterion for selection. Posting scholarly work to a preprint server increases its visibility and, hopefully, impact within the research community. It gets your findings into the hands of other scientists a bit sooner than normal – a bit closer to when the work was done rather than after months of reviews and revisions. It helps increase the “speed” of scientific discovery, as we learn about what’s new a little bit earlier than we would have from journals alone.

Here is the “why” answer from the ESSOAr FAQ page:

ESSOAr_banner_and_benefits

In addition to a lot of the same arguments I write above, there is an interesting comment in the middle of the paragraph, “You can establish priority.” Rather than the publication date being your time stamp laying claim so some finding, posting on a preprint server establishes that claim a bit sooner.

In a somewhat selfish consideration, the anecdotal evidence that I have heard is that posting your work on a preprint server increases the “early lifetime” citations to the paper. That is, it is thought that the page views and downloads of the preprint leads to faster incorporation of your findings in the work of other scientists, and citations to it therefore should begin a few months sooner. I am not sure how true this is, because the citation rate with year since publication is fairly constant at ~3/year in JGR Space Physics. Furthermore, I am told that the solar physics community extensively uses the arXiv preprint server, yet the journal Solar Physics has a Journal Impact Factor about the same or even slightly lower than JGR Space Physics. In support of preprint servers, I am told the astrophysics community uses arXiv even moreso that solar researchers, and The Astrophysical Journal has a JIF several points higher that the JIF for JGR Space Physics. So, perhaps my awareness of the solar community’s usage of that server is overestimated. This is all speculation, though; we need some quantitative statistics on usage and eventual citations to robustly claim anything. My point is that, while the evidence is mixed about the effectiveness of preprint servers, there is a plausible argument that they should lead to higher citations soon after publication.

Because it s really very little time and effort to upload, I think that it is worth it to do so. I suggest doing this when you submit to the peer-reviewed journal. I haven’t gone through it yet to see it for myself, but I am told that there is a link within the GEMS process for automatically sending the newly-submitted manuscript over to ESSOAr. The trickiest thing about submitting to ESSOAr was the license agreement. There are 4 levels of user licenses available to you. The most lenient is “CC-BY”, for which the only restriction is that users must properly cite it. For my Fall AGU poster, I selected the second level, “CC-BY-NC,” which places the additional constraint of no commercial reuse without my permission. The next level adds a restriction on “derivative use” without permission of the authors. The fourth one is the most restrictive and basically says it can be here on ESSOAr with no other use allowed. Aside from this, the process is very straightforward and easy.

The second step to achieving the full benefits of a preprint server is using ESSOAr as a place to learn about the latest results in your field. This requires signing up for new content alerts. Once you have logged in, conduct a search with some keywords of relevance to you. Once the results are up, then in the upper right area of the page is this:

ESSOAr_followresults

The first link, the magnifying glass with the plus symbol, will “save the search” for you. This opens up a new window where you can name the search and indicate how often you want it to automatically run this for you and send you an alert about it. It looks like this:

ESSOAr_savethissearch

The second symbol opens a page for setting up RSS alerts for the individual posters and preprints found in the search. Actually, both of these links are there regardless of whether you have signed in, you just can’t actually save the search until you log in.

On the page for each poster or preprint in the database, there are two links, “Track Citations” and “Add to Favorites.” The first allows you to get alerts on citations to that specific post, while the second just provides a quick link to that post. These settings, and the saved searches, can all be managed from your profile page. To get there, click on your name in the upper right corner and then on the “Profile” tab. On the new page that loads, the left-column menu has Alerts, Favorites, and Saved Searches.

There isn’t much content available yet – a handful of manuscript preprints and about 50 poster PDFs. If we all collectively start using it, though, then ESSOAr will blossom into a place where space scientists go to learn about the latest work being prepared for publication.