Preprint Servers: Challenges

A third (and probably final, for now) post on of ESSOAr, AGU’s new preprint server for Earth and space sciences. The first described it, the second touted it, and now this one is the ethical scold of how best to use it.

The biggest point to remember is that preprint servers are not peer-reviewed journals. Yes, there is an editorial board that checks submissions for scientific scope, but there is no vetting of the accuracy of the content. The editorial check takes a day or two, maybe a week max, but it is not a real review process. Yes, content here gets a DOI, but we should all remember that content on preprint servers are essentially just a step above “private communication” in terms of referencing authority. That is, it could be wrong.

We hope that content on ESSOAr, and any other preprint server, will eventually be published in a scientific journal. Researchers are putting their reputation out there with each new post on one of these servers, so the content is, for the most part, respectable. Go ahead and use it to learn what is being done by your colleagues. Because preprint server content has not been through the peer review process, though, it should be replaced with the “final” version of the study from whatever journal it eventually appears in.

To summarize in a graphic:

Caution-preprints

            Peer review should still be the standard for what is accepted as “knowledge” of the subject. Even this can be wrong but at least it has been thoroughly scrutinized by experts. You should be very skeptical of older preprints on the server (say, more than 2 years since original posting) that lack a link to a final published version of the paper. That work either was not submitted or did not pass peer review. If the former, then it is perhaps the case that the authors found a problem with the study and therefore never submitted that version of the paper. If the latter, then perhaps the editor or referees found a problem with the study and declined publication of it. Either way, the study did not reach its “final” form in the literature.

The advice to the community about older preprints can be summed up like this:

  • Authors: use caution when citing an older preprint.
  • Reviewers: pay extra attention to citations of older preprints.
  • Editors: ask reviewers to check the appropriateness of older preprint citations.
  • Societies: set policy about citing older preprints.

I am told that the astrophysics community, which regularly uses the arXiv preprint server, understands this difference in “publication” levels. That is, research communities can learn to use preprint servers and make it their go-to place for the latest content across a number of journals, as I am told that many in astrophysics do. They also know, however, that when it comes time to write your own paper, don’t rely on preprints as your main entries in the reference list. The astrophysics community, I am told, understands the guidelines about preprint servers and only uses it for finding the latest work on a topic.

We, the Earth and space science research community, should adopt this same mentality about preprint servers, not only ESSOAr but any server (and there are several being created). Such servers should be a place to get the latest studies from across a variety of journals, learning about content as the manuscripts are submitted rather than months later when they are accepted and eventually published. We should only use it for the latest work, though. A preprint server is not the place for full literature searches – those should be done in Web of Science, Google Scholar, Scopus, ADS, or other services that scan the published, peer-reviewed literature. And, as an editor, I strongly urge you to please conduct a full literature search, because a recent study by Mark Moldwin and me showed that the more complete your reference is, the more citations your paper will get (on average).

Use ESSOAr, but know its purpose within the hierarchy of scientific publications.

Advertisements

Preprint Servers: Benefits

With the launch of ESSOAr, AGU (and all of the other supporting societies on the advisory board) has entered the market of posting scholarly content prior to official acceptance by a peer-reviewed journal. Yesterday I discussed the “how” of ESSOAr, here I discuss the “why.”

The big reason is to increase scientific communication and collaboration. AGU’s mission is to promote discovery in Earth and space sciences, and many of the society’s honors, medals, and awards cite “unselfish cooperation in research” as a primary criterion for selection. Posting scholarly work to a preprint server increases its visibility and, hopefully, impact within the research community. It gets your findings into the hands of other scientists a bit sooner than normal – a bit closer to when the work was done rather than after months of reviews and revisions. It helps increase the “speed” of scientific discovery, as we learn about what’s new a little bit earlier than we would have from journals alone.

Here is the “why” answer from the ESSOAr FAQ page:

ESSOAr_banner_and_benefits

In addition to a lot of the same arguments I write above, there is an interesting comment in the middle of the paragraph, “You can establish priority.” Rather than the publication date being your time stamp laying claim so some finding, posting on a preprint server establishes that claim a bit sooner.

In a somewhat selfish consideration, the anecdotal evidence that I have heard is that posting your work on a preprint server increases the “early lifetime” citations to the paper. That is, it is thought that the page views and downloads of the preprint leads to faster incorporation of your findings in the work of other scientists, and citations to it therefore should begin a few months sooner. I am not sure how true this is, because the citation rate with year since publication is fairly constant at ~3/year in JGR Space Physics. Furthermore, I am told that the solar physics community extensively uses the arXiv preprint server, yet the journal Solar Physics has a Journal Impact Factor about the same or even slightly lower than JGR Space Physics. In support of preprint servers, I am told the astrophysics community uses arXiv even moreso that solar researchers, and The Astrophysical Journal has a JIF several points higher that the JIF for JGR Space Physics. So, perhaps my awareness of the solar community’s usage of that server is overestimated. This is all speculation, though; we need some quantitative statistics on usage and eventual citations to robustly claim anything. My point is that, while the evidence is mixed about the effectiveness of preprint servers, there is a plausible argument that they should lead to higher citations soon after publication.

Because it s really very little time and effort to upload, I think that it is worth it to do so. I suggest doing this when you submit to the peer-reviewed journal. I haven’t gone through it yet to see it for myself, but I am told that there is a link within the GEMS process for automatically sending the newly-submitted manuscript over to ESSOAr. The trickiest thing about submitting to ESSOAr was the license agreement. There are 4 levels of user licenses available to you. The most lenient is “CC-BY”, for which the only restriction is that users must properly cite it. For my Fall AGU poster, I selected the second level, “CC-BY-NC,” which places the additional constraint of no commercial reuse without my permission. The next level adds a restriction on “derivative use” without permission of the authors. The fourth one is the most restrictive and basically says it can be here on ESSOAr with no other use allowed. Aside from this, the process is very straightforward and easy.

The second step to achieving the full benefits of a preprint server is using ESSOAr as a place to learn about the latest results in your field. This requires signing up for new content alerts. Once you have logged in, conduct a search with some keywords of relevance to you. Once the results are up, then in the upper right area of the page is this:

ESSOAr_followresults

The first link, the magnifying glass with the plus symbol, will “save the search” for you. This opens up a new window where you can name the search and indicate how often you want it to automatically run this for you and send you an alert about it. It looks like this:

ESSOAr_savethissearch

The second symbol opens a page for setting up RSS alerts for the individual posters and preprints found in the search. Actually, both of these links are there regardless of whether you have signed in, you just can’t actually save the search until you log in.

On the page for each poster or preprint in the database, there are two links, “Track Citations” and “Add to Favorites.” The first allows you to get alerts on citations to that specific post, while the second just provides a quick link to that post. These settings, and the saved searches, can all be managed from your profile page. To get there, click on your name in the upper right corner and then on the “Profile” tab. On the new page that loads, the left-column menu has Alerts, Favorites, and Saved Searches.

There isn’t much content available yet – a handful of manuscript preprints and about 50 poster PDFs. If we all collectively start using it, though, then ESSOAr will blossom into a place where space scientists go to learn about the latest work being prepared for publication.

ESSOAr is here

The Earth and Space Science Open Archive is up and running. This is a new preprint server created under the leadership of AGU with technical and financial backing from Atypon and Wiley. Over a dozen scientific societies are participating in ESSOAr, advising on the development, structure, and policy of the site. It was announced in Eos back in September but it is now accepting submissions as of late January.

ESSOAr

            In the middle of the central graphic on the website is a search tool, to browse what is already uploaded to this pre-publication archive. There are several space physics posters already in the system. Yes, posters. If a meeting is approved by the advisory board, and pretty much all AGU-related meetings should be approved, then those that presented posters at the meeting will be able to upload a PDF of their poster to ESSOAr. That is, ESSOAr is more than just a preprint server but is also filling a unique niche in capturing the scientific content of the conference poster hall and allowing for a virtual poster session after the meeting.

To add content, you first have to log in to the system, which is done with your ORCID username and password. This allows you access to the author dashboard link along the top menu bar. The author dashboard gives you a second menu bar across the top, like this:

ESSOAr_author_dashboard

showing you all of the stages of submitting a poster or manuscript preprint to the server.

I went through the submission process with my Fall AGU poster and it is fairly easy. I think the trickiest thing was picking a license agreement for it. ESSOAr offers 4 levels of license. In the footer of all pages, there is a link to a nice Frequently Asked Questions list available about ESSOAr. One of those questions is about the license levels. It’s good to read this first. All levels require users to provide proper attribution back to this ESSOAr posting, but some prohibit commercial or derivative use. It’s also good to read through the fine print at the user terms and conditions.

Material posted here is not peer reviewed. There is a long list of researchers on the editorial board that check to make sure that submissions are within the scope of scientific endeavor appropriate for inclusion in the server. They do not, however, offer corrections or suggestions to the material. That’s on you.

Once something is posted, you can “revise” but not delete it or remove it. This is probably not needed for poster submissions, but manuscript preprints could be revised with subsequent iterations of the paper. Once published a link to the journal version of the manuscript should be added to the ESSOAr version.

Like preprint posts to arXiv, preprints put here are excluded from the similarity cross-check conducted on all manuscript submissions to AGU. In fact, AGU is making it easy for authors to upload to ESSOAr and then “transfer” some of the information you just typed over to a formal submission to an AGU journal. And the reverse, too: submission to GEMS will now have a link to also post on ESSOAr.

I’ll have other posts on the benefits and challenges of preprint servers. For now, happy posting.

AGU Centennial

The American Geophysical Union is turning 100 years old next year. The society has launched a major campaign to celebrate this triple-digit milestone of existence. They even have a nice logo:

AGU100_logo_V-RGB272-768x827.png

Details of the design of this logo are explained here.

There will be special events at both the 2018 and 2019 Fall AGU Meetings, one kicking off the festivities and the other wrapping it up. Note that the 2018 Meeting will be in Washington DC, with tours of the renovated now-net-zero AGU building. One of the big activities going on right now is the AGU Narratives Project, a joint activity with StoryCorps to record conversations about our experiences conducting Earth and space science.

JGR Space Physics is participating in the Centennial in several ways. Firstly, AGU is asking all of the journals to have a series of papers on Grand Challenges in their field. We are working on this. Secondly, we are making plans for a written version of the AGU Narratives project, a collection of papers from the pioneers of space physics. JGR Space Physics actually had a special issue on this exact topic over twenty years ago. There was also a book, a couple years later, entitled, “Discovery of the Magnetosphere.” We will be doing this again. We are also actively taking part in and coordinating with the AGU Centennial celebration planning.

To lead all of this, I have appointed one of the journal editors, Larry Kepko, to be the coordinator of our activities. He has fully embraced this role and is coming up with some good ways to have space physics to be integrally involved in the Centennial celebrations. If you have questions or comments about this, you can contact either him or me.

To make time for this, Dr. Kepko is pulling back a bit from the normal duties of being assigned “regular” submissions to the journal. I will still be assigning him a few papers, but far less than before. So, when you submit a manuscript, you can still request him as your preferred editor, but there is less chance that I will assign it to him because I am intentionally keeping his manuscript workload down.

This new role for Dr. Kepko, combined with a slowly increasing number of manuscript submissions over the years, is the need for adding two new editors to the JGR Space Physics board. The announced application deadline was yesterday (February 23), but you can still submit for a couple more days. I am off to the Editor-in-Chief meeting, which will occupy my time for the first half of next week. So, the deadline is unofficially extended until February 28. On March 1, when I am back in my office, I will start coordinating with the others on the search committee to begin the selection process. So, there is still time to apply for this position. If you have any questions, then please send me an email, or contact any of the current editors.

Annotating Manuscripts with Hypothes.is

A few months ago, AGU introduced a new feature in GEMS – annotating the merged PDF of the manuscript. Senior AGU Pubs staff wrote an Eos Editors’ Vox article about it. AGU has partnered with hypothesis.is, an online annotation tool, so that reviewers can highlight text and insert comments. Editors can then add additional comments before making a decision about the paper. The comments are labeled “reviewer 1,” “Editor,” etc., so that the author can identify which of the assessors made the remark. During the revision process, authors can respond to these comments directly in the annotated PDF.

hypothesis_logo

            I have used it a couple of times and I have seen ~10 reviewers use it over the last few months. I think it works really well, so it is it time to publicize this feature and make the community aware of this powerful resource.

When you agree to review a manuscript, you will see this new section on the review page:

hypothesis_entry_button

It’s just below the link to retrieve the paper and the link for submitting your review. When you click on it, you get a new browser window with the manuscript PDF:

hyothesis-screenshot

This page already has several sections of text highlighted with example comments written. There are controls across the top bar for navigating around the document. When you highlight some text, a small pop-up window appears below it with the word “annotate” in it:

hypothesis_annotate

This opens a text box in the right-hand column in which you can type your comment:

hypothesis_textbox

The “You” at the top indicates the originator of the comment, then the highlighted text is repeated, and then a box for writing your comment, including rich text features like inserting hyperlinks, images, and LaTeX-based equations. Along the bottom of the text box is a row of buttons for specifying the type of remark you are making. Is it an overview comment? Pick “Summary.” Do you want to designate it as a “major” or “minor” concern? Go for it. Are you suggesting a small English usage correction? Then pick “Edit.” Are you suggesting a new reference or two, or commenting on a figure? Click that button, then. Finally, there is a “Confidential?” button that you can click if the remark is just to the Editor and not meant for the author. I promise to look through the comments and read these.

Back on the main reviewer page, you can actually see if there are annotations on the “annotated merged PDF.” It should appear as a new link, “Show Summary Table,” like this:

hypothesis_summary-table-link

When you click on this, all of the comments in the hypothes.is PDF are shown:

hypothesis_summary_table

Nice, huh?

Note that you still need to click the link on the main reviewer page to complete the review:

Review_the_manuscript_button

You should answer the pull-down-menu questions and fill in any comments you want in the review text box. It is helpful if you, at the least include a sentence like, “Please see my detailed comments in the online annotated PDF.” This reminds the Editor to go to the annotated PDF and see your comments there. It is also helpful to include a short paragraph summary of your review there. In fact, you can make your review a hybrid of the two, with major comments in the review text box and specific comments embedded in the annotated PDF.

In addition to the Eos article and this blog, there are also more detailed author instructions, reviewer instructions, and even editor instructions at the AGU website. The hypothes.is website also has a really good tutorial. Also, one caveat: it is an interactive web-based tool, so you have to be online to use it.

Also, this whole thing is optional. You don’t have to use it. So far, I’d say that most reviewers do not use it. But most reviewers could be using it, so please consider it. Many reviews include line-identified comments, and this new feature should be easier than typing the location coordinates into your review.